Vision-Based Diagnosis and Treatment

ABSTRACT

Devices and methods are provided herein for aiding in the diagnosis of and response to conditions ranging from unpleasant to disabling. Images are presented to viewers and eye-tracking of their eyes during that process along with software analysis non-subjectively identify and quantify complex visual signatures formerly hidden in the responses of the eyes. This allows caregivers, when presenting these images to people to be evaluated for a condition, to identify, again with eye-tracking and software analysis, those signatures indicative of a presence or an absence of the condition. An associated remediative response provides treatment applicable to a real-time environment.

This application claims the benefit of the provisional application U.S.Ser. No. 61/800,511 filed Mar. 15, 2013. That application is entitled“Vision-Based Diagnosis and Treatment.”

This application also claims the benefit of the subsequent utilitypatent application U.S. Ser. No. 14/215,167 filed on Mar. 17, 2014. Thatapplication is also entitled “Vision-Based Diagnosis and Treatment.”

Both of these applications are referred to and incorporated herein byreference in their entirety.

BACKGROUND OF THE INVENTION

Strabismus, sometimes referred to as “lazy eye,” is a medical conditionof the eye normally treated by disabling or otherwise interfering withthe vision of the strong eye so that the patient will be forced to usethe “lazy” one. Although the current treatments for Strabismus(chemically paralyzing, numbing or blocking the strong eye with anopaque patch) are unpleasant and, consequentially, irregularly applied,they can be effective if they are used consistently. Their effectivenessas a normative treatment for Strabismus appears to be related to thebrain's ability, under external duress, to recognize and correct certainfactors that result in poor vision.

Autism, a pervasive developmental disorder recognizable by physicalrigidity, emotional detachment and impaired communication is both amajor and a growing threat to children. According to a study publishedin Pediatrics, Oct. 5, 2009, based on a National Children's HealthSurvey done with 78,000 parents in 2007, 1 percent of the population ofchildren in the U.S. ages 3-17 have an autism spectrum disorder. It isalso the fastest growing developmental disability (10-17% annual growth)according to an Autism Society estimate based on 2003 US stateeducational data.

Both disorders are especially prevalent in children, desperately needearly diagnosis for a successful outcome and share a dearth of effectivetreatments whose side effects, discomfort, tedium and cosmeticdisincentives to compliance do not require more patience and socialconfidence than this young and already emotionally challenged populationcan live up to.

Eye-tracking is a broadly used technology to determine the vision axesof the eyes. Then, responsive to their position and orientation, it canbe determined essentially where the subject is looking. There are a widevariety of technologies for tracking the vision axis of each eye, all ofwhich are applicable to the current invention.

For example, Mason in U.S. Pat. No. 3,462,604 on Aug. 19, 1969 uses anoculometer (a device that records the differences in electrical chargebetween the front and back of the eye. This can then be correlated witheyeball movement).

Graf in U.S. Pat. No. 4,109,145 issued Aug. 22, 1978 uses an oculometeror any other line of sight determining device and measures the length ofstatic fixation. If the time of fixation passes a threshold value, theapparatus produces a control output (it's considered a valid fixationrather than an unintentional saccade).

U.S. Pat. No. 3,724,932 issued to Cornsweet et al. Apr. 3, 1973 uses aplurality of Purkinje images from the reflective surfaces of the eye.Monitoring the separation of the Purkinje images indicates theorientation of the optic axis of the eye.

U.S. Pat. No. 4,866,229 issued to Scharfenberg on Sep. 12, 1989 uses aheads-up display to track the eyes while the heads-up display is worn.

U.S. Pat. No. 4,651,145 issued to Sutter on Mar. 17, 1987 usesoculo-encephalographic signals captured responsive to unique codedsignals presented to the subject with the EEG signal then used todetermine where the subject is looking.

U.S. Pat. No. 5,293,187 uses electro-oculogram signals to control videodevices.

Knapp et al. in U.S. Pat. No. 5,293,187, issued Mar. 8, 1994, which“relates generally to the operation of three-dimensional games anddevices and strabismus measurement by determining the independentposition of each eye” used an electrooculogram (electro-oculogramsignals are, in effect, an electrical signature of eye movement that isnot sensitive to ambient light interference) to determine eye positionand to determine the horizontal and vertical position of each eye aswell to determine convergence or divergence of the eyes. The signalsrepresenting eye position are interfaced to an output device forstrabismus measurement. It is for diagnostic purposes only

BRIEF SUMMARY OF THE INVENTION

It is an object of the current invention to provide both a means forearly diagnosis of autism and a means for its treatment that encouragesremediation in even the youngest of patients.

It is another object of the current invention to provide a means forearly diagnosis of strabismus and a means for its treatment thatencourages remediation in even the youngest of patients.

Is another object of the current invention to provide a real-timeresponse to recognized conditions that is both timely and graduated toenable minimally distraction and immediate patient comprehension of aneed for correction, a recommended magnitude of correction and, whereapplicable, a recommended direction of correction.

It is another object of the current invention to be in an adequatelycomfortable and cosmetically acceptable form conducive to long-termapplication periods and patient compliance for these and otherconditions recognizable by their ocular orientations and movements.

It is another object of the current invention to integrate other stimuliand recognized patient symptoms into an improved diagnosis and responsethat benefits from multiple presented indicia.

It is another object of the current invention to execute algorithmsresponsive to real-time data over time to identify trends and conditionsand both alert and aid physicians in treatment and analysis as well asprovoke remediative user response by one or a plurality of systemresponses.

It is another object of the current invention to self-generate a set ofscoring criteria responsive both to the visual responses of viewers withknown and well established levels of the condition in question and toage and other key indicator factors applicable to making the currentinvention more precisely tuned to specific patients. Thus the currentinvention is applicable to self-scoring for general research (applicableto discovering and scaling new vision indicators) and to a continuallydeveloping remediative program for any condition with predictable andnormative visual responses.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a viewer, 101, observing a displayed image on ascreen, 102, at a point 103. The images are being displayed by computer,109 as the eyes of the observer are being tracked by the eye trackingassembly 108.

FIG. 2 is the assembly of FIG. 1 wherein the convergence of theobserver's eyes occurs at a point, 107, distal to the screen, 102.

FIG. 3 is a flowchart illustrating exemplary steps in the process ofexecuting the devices and methods of the current invention.

FIG. 4A and FIG. 4B support the verbal illustration herein of apath-finding game comprising at least a maze, 401 and a goal 402.

FIG. 5 illustrates a worn embodiment wherein the eye is tracked bycamera, 502. The worn assembly may include a forward view camera 504.

FIG. 6A and FIG. 6B illustrate the basis of another path-finding game.

FIG. 7A, FIG. 7B, FIG. 7C and FIG. 7D detail exemplary databasestructures useful to the discussion of one embodiment of the currentinvention.

FIG. 8, FIG. 9A and FIG. 9B support the discussion in the specificationof an exemplary process for scoring the values responsive to theimportance of the observer looking at different points on the screen.The center points of the two exemplary sets of concentric circles shownillustrate two exemplary points on the display associated with a giventendency. The distances between each such point and a pointrepresentative of the observer's instant point of focus are related tothe score.

FIG. 10A, FIG. 10B, FIG. 10C, FIG. 11, FIG. 12A, FIG. 12B and FIG. 12Clist programmatic code related to scoring user responses and identifyingparameters best suited to the purpose.

DETAILED DESCRIPTION OF THE INVENTION Autism

It is not the purpose of this description to describe or attempt tounderstand all the mysteries of autism many of which may continue to bemysteries for generations to come. Nor is it the intention of thecurrent invention to solve all the problems associated with thewidespread and rapidly advancing disorder. However, a number of oculardynamics have been observed in the body of research to be normative ofand often peculiar to those with autism. A few of the well knownexamples are that those with autism are highly prone to avoid eyecontact (but able to look into the eyes of animals), fail to followmotive-interpreted actions and suggestions as well as to stray theirinstant point of interest (POI) to areas of no interest at all to otherpopulations.

The POI can be any value relative to where the viewer is looking, e.g.,along the cyclopic vision axis (and for some applications this won'teven be on a screen) but, for a viewer looking at a screen, the POI canbe thought of as the screen coordinates identified by the eye-trackingequipment for the point on the screen where the viewer is currentlylooking. (The cyclopic vision axis, a descriptive convenience, describeshere an imaginary vision axis extending from the midpoint between thecenter of the two eyes, where the eye of the mythical Cyclops was, tothe intersection point of the two vision axes when looking straightahead. For example, when the viewer is looking straight ahead, thecyclopic axis follows the intersection of the sagittal plane and theparticular transverse plane that intersects the center of the eyes thusforming an imaginary but useful descriptive device for identifying netsingular direction of dual-eye subject fixation. When the subject looksaway from dead center, the cyclopic axis rotates and continues to bisectthe angle between the two real vision axes).

Because of the aforementioned difficulties of diagnosing autism as earlyas possible and providing extended treatment very early and for longperiods to an extremely impatient demographic, the current inventionprovides a device and method for recognizing it at a very early age andwith a statistically reliable diagnostic process and treating it with auser-friendly real-time responsive system.

In one preferred embodiment, the subject, 101 in FIG. 1, looks at avideo screen, 102. The source of the image on that screen can be fromany source that can be associated with a set of values including networkTV, cable, DVD, jump-drive, hard drive, streaming video, etc. However,in the embodiment shown, a personal computer, 109 provides a video imageto the display, 102 from a video stored on a hard drive local to thecomputer, 109. An example session of one such embodiment is nowexplained essentially following part of the flow chart of FIG. 3.

The user interface, with user choices normally entered through thekeyboard responsive to screen prompts, allows function selections, oneof which is video selection. After any source and format of video isselected, the subject may choose to run the selected video (shown in thepost video selection decision block of the flow chart in FIG. 3 as “RunY/N”). If the selection is “Y”, the program proceeds to begin loadingthe video and at least the first frame's (because each frame in a videois an independent image, a frame will sometimes be referred to herein asan image) corollary data. The paragraphs below are numbered with steplabels identifying steps on the flow chart of FIG. 3.

A Loading Video and any Corollary Data:

Data identifying, for a short time period, scoring values for specificareas of the image, can be read from a storage device (e.g., a discfile) at the beginning of the process. For example, in step A of FIG. 3all of the data for the entire video can be read into memory and usedwhen needed. Or, of course, the data can also be accessed from thedatabase table or other form of storage for each time period in thevideo as that time period comes up (this latter approach is the onedescribed herein). It can even be stored in the video blanking intervalof a television program. Any increment of time can be used to break thevideo down into convenient component parts. In the example embodimentbeing described here, frames are used as the minimum time segment. Forexample, if the video executes at 30 frames per second, each framerepresents 1/30 of a second. Multiple frames can then be grouped intovignettes of many contiguous frames which may be representative of aunique purposeful interlude and vignettes will often be scoredindependently to facilitate comparison of specific vignettes of activitybetween populations and known norms.

Then, as the prescreened video is being played, the thus retrievedscoring values for the instant frame are used to tabulate scoresresponsive to the importance (identified by the scoring values) of a POIoccurring in prescribed areas.

More detail on what this involves is discussed further below. In thisexample embodiment, at minimum, the video file loading begins and atleast the first frame is prepared for display. Thus, the softwareexecuting on the computer, 109, reads a video file from any storagedevice and displays at least the first frame on the display, 102. (Somealternative control software embodiments do not call or commandindividual video frames but otherwise synchronize scoring logic with thecurrent frame being displayed and the current positions of the eyes.However, this example describes an embodiment that does.)

Also, corollary data, including data unique to the current frame ofvideo, can now be retrieved from any storage means (a computer harddrive in the preferred embodiment).

Although the same corollary data can be used for both diagnosis andremediative treatment, they may also differ such as in level ofcalculation overhead required. For example, during a diagnosis cyclegreater precision paid for by increased computer calculation overhead ispractical even on a slow processor since the entire diagnosis can beprocessed on the back end as a batch process. However, real-timeresponsive remediative action and/or execution on a slow processor mayresult in slightly more time-efficient implementer choices for bothcorollary data values and procedures. Also, the corollary data examplesshown below are merely exemplary of one of the many applicableembodiments of the current invention.

For example, corollary data (non-video image data) may include elementssimilar to or including the exemplary set in the data structure below.

Data Structure and Vignette Organization:

One of many applicable software application processes to manage thisparticular optional scoring technique is for implementers to firstestablish the range of frames that make up a vignette that may or maynot be part of a considerably longer continuous presentation. In thepreferred embodiment, the presentation is made up of a series ofvignettes each with their own scoring attributes. In embodiments wherethis approach is used, the following data structure describes thegeneral nature of a preferred format for a database stored on a computerhard-drive which may be loaded into memory prior to video viewing forfaster access.

The primary database includes for each frame (although it is not arequirement, here we provide an individual database record for eachframe) a value for at least frame number and vignette (frame-grouping)number. Although there can be multiple primary database records for asingle frame, the preferred database structure makes that unnecessary asit is characterized by a one-to-many relationship between the primarydatabase just described and a secondary database having one record foreach cue rule for the frame that is represented by the instant singlerecord of the primary database (here, using the frame number as theindex key that relates the two tables). Here, the single primarydatabase record for each frame can point to many records in a separateor corollary second table containing cue rules for the current record inthe primary database (which refers to a single frame). In this preferredone-to-many relational database example, there is, of course, no need tostore cue rules in the primary (first) database since they will be inthe related secondary database. All alternative database approaches(e.g., SQL on-the-fly calls and pre-sequenced serial tables) areapplicable to the current invention and obvious to those skilled in theart.

Example Primary Database Fields:

Parent (Primary) Table: ScoringPrimary

-   -   1. FrameNumber: (Integer) This is the number of the video frame        this database record refers to. Where there is one primary        database record for each video frame (this is the presumption        for this example), this can optionally be replaced by the        “record number” value maintained by many database engines.        Alternatively, for slower processors or slow-action video        segments requiring less time resolution, this frame number field        may be used to represent a frame group number (for example one        frame group number for every F frames). In that alternative        case, complete processing would only be executed once for every        F frames.    -   2. VideoReference: (Integer) This is a placeholder field        representative of any related field data implementers use to        provide information for and sync frame display with a given        frame. While the current invention does not require any        particular one of the various video control and synching methods        (all are applicable), in an embodiment where a specific video        frame is called for a given record in this database, this number        may be used to identify that video data for display.

Of course, a third database related to the second could also be used formany of these values that are used many times (but is not in thisexemplary description). For the related (secondary) database, we usehere the corollary data fields already described above.

Child (Secondary) Table: ScoringChild

FrameNumber (Integer). This is the index key between the parent(primary) and child tables.

VignetteNumber (Integer). This identifies and groups a range of framesfor which this particular meaning attribute (below) applies and willresult in calculations for.

This number is also used to recognize the beginning and end of eachvignette simply by when this number changes. Typically, scoring will bedone at the vignette level and some or all of vignettes' data will beincluded in an overall summary report.

MeaningAttributes (Integer). This identifies the symptomaticcharacteristic whose tally will be affected by the score. (Note that asingle meaning attribute can result in adjusting more than one tally).This can, of course, alternatively be a string value (rather than aninteger) descriptive of a meaning which may be helpful in applicationswhere there is a small number of meaning attributes. However, in thepreferred embodiment the “handle” for a meaning attribute (such as“interpersonal eye engagement, opposite sex and adult”) is a uniqueinteger.

TargetCoordinates (String, 10). These values identify the location onthe screen of a direct target “hit” for the meaning attribute for thischild record. Example: “0012300777”, in a 2-D implementation, representsthe screen location 00123,00777). These can be pixel numbers in a rowand column format as is common or any other location identificationstrategy. Herein, the left set of values will be referred to as X (orcolumn) values and right set as Y (or row) values. Target coordinates,as well as other field values below, may be entered by reviewerassistance software as described below.

TargetStrikeValue (Single Precision Decimal, 3). This is the basenumerical scoring value for the viewer fixating precisely at the targetcoordinates. Admittedly, three decimal points seems excessive for thebase calculation value for a “direct hit” within the target full valueradius defined below. However, when the current invention's “MonteCarlo” process for scoring optimization is taken into account, thislevel of precision can, particularly in second and third levels ofdiscretization processing, be of great value.

TargetFullValueRadius (Integer). This defines an optional tolerancecircle for “full credit” of the target strike value. It indicates, howfar away the POI can be from the target

coordinates and still obtain the full target strike value score Actualdistances between the POI and the target strike value will, of course,include non-integer values. For example, identifying left-rightscreen-location values as X values and up or down values as Y values,the distance to the target coordinates is calculated as:

D=((Xt−Xp)²+(Yt−Yp)²)^(1/2)  a.

where Xt and Yt are target coordinates and Xp and Yp are POIcoordinates. While this will certainly result in non-integer values, thelevel of precision required, particularly where location values aredenominated by small pixels (for example where the target full valueradius is expressed as a the number of pixel widths between POI andtarget coordinates), will normally be adequately met with an integervalue for this field.

Additively or alternatively, the target full value radius can beaugmented or replaced by target full value width and target full valueheight for similar operations using a rectangle centered at the targetcoordinates.

TargetFullValueWidth (Integer)

TargetFullValueHeight (Integer) Where rectangular (rather than circularas used in target full value radius) area containment for full value isdesired, these width and height values provide the means to determine ifthe POI falls within one half of the width value to the left or right ofthe target coordinates and within one half of the height value above andbelow the target coordinates by simple subtraction of Cartesiancoordinate values of the target coordinates and the POI as is widelyunderstood.

TargetFullValueEllipseA (Integer) Where an elliptical area ofcontainment is desired, it can be determined if the POI falls within anellipse having a major axis twice the magnitude of the minor axis. UsingX′ as the absolute difference in x between the POI and the targetcoordinates and Y′ as the absolute difference in y (vertical on thescreen) between the POI in the target coordinates, the maximum value forthe magnitude of the POI y coordinate is obtained using the equation:

Y=a/2*sqrt(1−(x̂2/â2))

thus, if Y′>Y, the POI does not fall within the target full value areaand will not be given credit for falling within this area when scoring.Other embodiments will certainly add a second value responsive to theminor axis thus enabling more variety.

Implementers may choose to use circles, rectangles, ellipses, any numberof other applicable shapes to determine if the POI falls within an areaclose enough to the target coordinates. For the sake of brevity, thecircle is used as the example to be explained in depth for most of thediscussion herein. However, the fields are provided such thatimplementers could use one or any combination of the shapes asindividual scoring elements each contributing individually to the scoreif desired.

SpatiallyDeprRespCurve (Integer) This is the optionally formulativescore-depreciation response to distance between the target coordinatesand the viewer's actual instant POI. Although formulae could be includedas a field value here were this a string field, in the preferredembodiment this field is simply an integer representing a formulaaccessible by the software and identified by this integer.

SpatiallyDeprRespRange (SDRR) (Integer). (SDRR) This is the furthestdistance a POI can occur from the circle associated with the target fullvalue radius and still result in any score at all. In other words, thedepreciating response curve will not be applied for POI's whose distancefrom the target coordinates is greater than the sum of the target fullvalue radius and the SDRR.

TimeFactor (Integer). This is an integer identifying the sequence numberof a calculation for augmenting scores responsive to the persistence ofthe fixation. The calculation that the software identifies with hisinteger includes the minimum and maximum number of contiguous frames inthe same vignette having the same meaning attribute and a scoremagnitude above the same score threshold (described just below) thatwill be used to raise the magnitude of the score responsive to thepersistence of the attribute. The calculation so identified will oftenbe formulaic in order to most realistically value persistence ofbehavior and to provide variables whose coefficients, particularly whenidentified through the Monte Carlo process, best reflect the nature ofthe condition being diagnosed and treated.

ScoreThresholdPositive (Single Precision Decimal, 2). This is thepositive score magnitude above which the time factor may be used tofurther increase the score),

ScoreThresholdNegative (Single Precision Decimal, 2). This is thenegative score magnitude above which the time factor may be used tofurther decrease the score), and

SkipFrames (Integer). This is an optional integer indicating how manysubsequent frames will continue to follow this rule and increment scoreswith it without having to reread the data file again). This defaults tozero. When the skip frames option is used (when it's value is positiveand non-zero), it is not necessary to make or load here subsequentdatabase records for this meaning attribute (up to the number in thisfield) (making database file sizes smaller and enabling fasterprocessing).

Condition (Integer). This optional value is the condition of the subjectupon which this scoring record was based

AnchorPower (Integer).

Multiple values for any and all of these corollary data will benormative allowing in any single video frame any number of identifiedtarget locations with individualized scores and/or responsive actions.The actual scoring process will be discussed further below.

B Viewer Analysis and Eye-Tracking:

There are wide variety of eye-tracking systems. Some, for example, havea single camera or two laterally-separated cameras either located oftenbelow the screen (to avoid eyelash interference) as shown in 108 ofFIG. 1. Some use facial characteristic algorithms to recognize headpositioning, etc. and, in the end, return values that can be read as orcalculated to be representative of POI. In the flow chart of FIG. 3 thisviewer analysis is re-performed for each frame to most preciselycalculate, in addition to other measures, POI. However, for a seated orreclining viewer, this step can, by implementer or user choice, bereduced substantially to once every so many frames to reduce processingoverhead.

Also, for improved time-efficiency, every execution of this process neednot wait to be begun in the series order of FIG. 3 which is simply oneexample procedure. For example, on the first pass, the initial subjectposition, orientation, distance to screen and initial POI data may becalculated earlier and in the background (simultaneously with otherprocesses like frame image and corollary data loading or, for example,during any of the scoring steps).

This process, which normally includes reading of viewer position fromthe camera image(s) of the subject, can actually be an optional featurein some embodiments of the current invention. For example, in lab testsit was adequate for some applications just to have the subject sitreasonably still in a known position (that is, at a known distance fromthe screen and sitting essentially centered preferably with the cyclopicvision axis, when normal to the coronal plane, approximatelyintersecting the center of the screen). From this approximate positionaldata, the angles of the eyes and thus their POI can be approximated in amanner that is adequate for some applications and implementerpreferences.

Thus, for such embodiments, the subject may approximately center his orher (the masculine identity will be used herein for brevity) headposition in front of the screen at a known distance from his eyes to thecenter of the screen and either enter, via keyboard or other data entryprocess, the approximate distance or accept an approximated defaultvalue. From the known position of the camera or cameras imaging the eyewith respect to the center of the screen, the position of each eye withrespect to each camera may be calculated by ordinary triangulation.There are many well-known and applicable methods and devices forcalculating the subject's POI from this data and the captured images.Without detailing each here, the location of the subject's fixation isidentified.

Additional Calibration:

Often even greater precision and more applicability to broader uses arebenefited by this positional data being accurately calibrated. For thisfurther improvement, a calibration session and calculation can beexecuted ideally prior to watching a video. These eye-tracking andvergence calibrations are known to those skilled in the art and may beas simple as displaying a small circle at a known location (e.g., acorner or at center) on the screen and prompting the viewer to lookthere. Typically, the software then creates a calibration curve relatingany eye-tracked POI to the calibrated value.

There are, of course, alternative or contributing ways to calculatesubject position that are applicable to the current invention includingbut not limited to dual laserbeam convergence, single angled laser beamposition, laser Doppler, projected facial grids, sonar, facial featureand feature-placement (eyes, nose, etc.) recognition, etc.

Also, and applicable to both subject positional data capture andlocation of POI, any alternative means of eye tracking or line of sightindication are applicable to the current invention including but notlimited to electro-oculogram signals (particularly for the fast framerate) and oculo-encephalographic sensing (where the displayed videoimage itself or components thereof can serve as adequate opticalstimulus). All available elements effective for measuring and reportingthe positions, or angles, or points of vision-axis intersection (or anycombination of these) are understood to be included as applicablealternatives when the term “eye tracking” is used herein for.

In one preferred embodiment, the eye tracking or other line of sightobtaining component chosen by implementers will rapidly and accuratelycapture the data needed for step B from a plurality of captured cameraimages often enhanced by additive lighting and powerful softwareanalytics. For example, the Tobii IS-2 Eye Tracker from Tobii TechnologyAB Karlsrovagen 2D Danderyd Sweden, provides an OEM (original equipmentmanufacturer) ready module mountable at the bottom of a video displayscreen and having a published frame rate of 30 Hz with accommodation forsubstantial subject movement without loss of accuracy.

C. Locate Apparent POI:

Eye tracking equipment, including but not limited to those alreadydiscussed, now identifies the POI. In the applications for diagnosis andremediation of autism, this includes the identification of a point orarea on the screen.

If the viewer's fixation is not anywhere on the screen, the point ofactual eye intersection may be recorded as a virtual position outside ofthe Cartesian coordinates of the screen but on the same Cartesian planewith respect to the center of the screen as the origin to simplifyscoring calculations. While corrections for off-screen discompliancecould be effected at this point, in the preferred embodiment, thisnow-identified POI is captured for action in later steps. However, insome applications, e.g., strabismus, the viewer's vergence may, in fact,result in a fixation that is significantly more proximal or distal tothe viewer than the plane of the screen and this is applied usefully bythe current invention as indicated herein.

In the preferred embodiment, this POI location is stored as the valuedescribed as the target coordinate in the data structure above. Thus, anX,Y pixel location on the screen itself is thus identified.

D. Disorder/Normative Scoring

There are vast differences in the way different implementers scoresubjectively interpreted phenomena and, thus, how they will implementthe current invention. Thus, providing examples as we do here, whileshowing a few approaches, probably does a better job of showing how verydifferently different researchers compile and analyze data. Nonetheless,for one example, let's establish a short vignette for individual scoringmade up of a potentially short range of video frames in a longer video.This may be part of a longer continuous presentation but may be scoredseparately as a vignette. Scoring from multiple vignettes can be used tocreate a combined score. In the short vignette, we will score based on asimple set of scoring values.

Pre-Screened Material:

In one embodiment of the current invention, videos are prescreened byknowledgeable personnel who identify areas in the image and provide forthem particular attributes and values responsive to the condition theyseek to diagnose and treat.

However, because it is tedious for reviewers to locate and type in longtarget coordinates, for every frame, implementers will typically includea time-saving user interface for prescreening personnel that allows themto identify an area with a circle or other shape over the area ofinterest in the image being reviewed with a mouse or other pointingdevice. Complex shapes may be used in applicable embodiments but thisexplanation herein uses a circle around the target coordinate locationwhose radius (the “target full value radius” field described below)defines the area around the target coordinate location at which a POI inthat area will be treated as a direct hit (i.e., as if the POI exactlyequals the target coordinates).

The software will then identify the central target coordinate location(preferably a weighted “center of gravity” for more complex shapes usedin other embodiments but in this example simply the center of the circledrawn over the area of interest on the screen showing the frame by thereviewer) and automatically create a record in this table. That recordwill have a frame number value driven by the actual frame number beingviewed by the reviewer, a target coordinate value driven by thethus-calculated center of the area chosen by the reviewer, and a targetfull value radius (described in the data structure above) based on theradius of the circle circumscribed by the reviewer.

The reviewer will then be asked to key in or otherwise enter the otherdata field values below and will simplify the process by making thedefault value either the same as for the previous record. Whereapplicable and for faster entry, the default values for the fieldslisted above, which can be overridden at the keyboard, can be animplementer-chosen value based on the software-recognized nature of thearea in the image selected by the reviewer (such as a human eye).

When these are prescreened, the screening personnel assign values fordifferent ocular behaviors for a given frame or frames, based on thedisplayed actions and elements that occur in those frames, by entering,in the preferred embodiment, data into a database or table for valueslike or similar to those listed above in the data structure for anexample embodiment.

Vignettes:

The beginning and end of a vignette is recognized by a change in thevalue of the vignette frame-grouping number (a field in the primarydatabase). Thus, (in this particular example of a scoring style) foreach frame in each vignette, the software will not only read thedatabases for the scoring criteria related to the instant record in theprimary database (which is responsive to the current frame being scored)but will zero a count variable for each score element at the beginningof execution for each vignette and increment these variables as thevignette proceeds for each credit amount.

A vignette made up of a group of contiguous frames can be a very simplescoring subset of a longer video made up of many sequential vignettes.For example, a series of frames may simply show a scene where a persontesting positive for the condition being tested is likely to look duringthis little theatrical vignette. If the POI occurs in the areas soindicated in the contiguous records having this same vignette number(which defines the length of the vignette), a positive score for thevignette will result.

Multiple simultaneous vignettes can also be used at the same time, evenin the same frame to individually score different areas with differentscoring attributes.

Scoring Logic:

Since it is difficult and normatively less frequent for those withautism to look at human eyes that appear to be looking at the camera(and thus, perceptively, at them), the area very proximal to the eyes ofsuch a person in the image of a video frame may be coded with anoptionally negative value for normalcy. (Obviously, positive valuescould also be used for normalcy on a similarly applied scale. Bychoosing to use the negative end of the scale for normalcy, we choose touse positive to indicate a disorder. Thus, a positive result will beunderstood to indicate the presence, and optionally the approximatemagnitude, of the disorder condition recognized.) Thus, if, in thisexample, the viewer looked directly and precisely in this normalcy area,that viewer would receive a higher magnitude (in this chosen example, amore negative) symptom score.

If the viewer looked in another area substantially away from thisanticipated fixation area or in a highly positive area (e.g., far fromthe normative viewing area and potentially in a highly non-normativeviewing area), the implementer could choose to rate that second areawith a score more towards the positive. In the preferred softwareembodiment and reviewer procedure, this second area in the instantframe's image is scored with the use of a second vignette even for thissame frame or group of frames.

3 Simultaneous Vignette Example:

Thus, consider an even more complex frame in which the reviewer wants toscore the frame using three different areas with their own criteria forscoring. This will be done, in the preferred embodiment, with oneprimary database record for the frame (and optionally any video syncinginformation) and three related child records (related by the framenumber field that is in both databases) each of the three records havinga different vignette number.

For example, the first vignette can be dedicated to an area of the imageassociated with highly negative scoring such as the screen location ofthe eyes of an angry person glaring at the viewer. When the viewer looksdirectly at this area, it is highly non-positive for the dysfunctionalcondition being tested for and thus has a negative score. Normally, aviewer will not be looking at two disparate areas of the same framesince there are typically 24 to 30+ of them per second, so only one ofthese three vignettes will receive a score for this frame.

Let's let the second vignette for the same frame regard an alternativearea on the same image frame that the reviewer believes should testpositive for the condition being tested. Perhaps the scene is set upsuch that, given the choice of a glaring set of eyes from an authorityfigure (covered by the first vignette) and the welcoming eyes of acontextually irrelevant dog far from the central areas of action (thesubject area of this second vignette), the significantly autistic personwill be looking at the dogs eyes. Thus the child database record thisframe and with this vignette number will calculate a positive scorebased on the degree to which the viewer POI fixates upon and persistsupon this area.

For the third vignette, representative of the many possible simultaneousvignettes, consider a more neutral area of the image that isnonnormative for healthy viewer fixation. A POI in this area could bereviewer chosen to have a moderately positive scoring value.

Thus vignettes can be used in combination to score individuallyessentially every area of the screen either positively or negatively.

in this preferred embodiment, if there is, for example, an image areawith a negative score in the same group of frames where the revieweralso wants to code another area for relatively positive (indicative ofdysfunction) score,

Time of Fixation

is also a substantial optional criteria applicable to the scoringprocess. For example, up to an implementer-chosen amount of time, thelonger the subject looks at that person's eyes, the higher the resultingnormalcy score based on scaling values chosen by the implementer. Justfor example, to minimize false negatives, an implementer might choosefor the software to return zero or only slightly negative values forhealthy fixations of less than 5 frames (or perhaps ⅙ second).

Action and Topical Focus:

For implementers seeking to capitalize on published reports that, in anaction frame such as someone pointing frantically in a direction, aperson without autism will tend to look at what they're pointing outwhile a person with autism may instead look somewhere else, thoselooking at the point of visibly indicated action would receive a highernegative score.

Multiple combinations are obvious as are any number of conditions thatan implementer may desire to include in the scoring process. As timegoes on, implementer opinions about different stimuli will certainlychange and thus the scoring values for areas of a frame will change withthem. Thus it is not the place of this discussion to limit the currentinvention to current numerical values, titles, or criteria for anystimuli or phenomenon but, instead, to provide devices and methods thatimplementers can use to score responses to stimuli and to use thesescores to diagnose conditions and implement remediative actions.

Example Scoring:

An example follows to explain several of the fields in the exampledatabase. It is very likely that many if not most reviewers will useonly a few of the fields in any given vignette to fit their particularneeds for that subset of the video. However, in this example, we willstretch the example a little to allow all of the fields to be used inthe corporate scoring of many elements for the same frame. The databasefields, format, interface from fields to video (or the absence thereof),and even storage elements may and will be changed by implementers withtheir own objectives from those discussed here. This is merely anexample of how the current invention can be used.

Just to set the stage to help explain this process, we describe a videowhose first 20 frames are innocuous enough and have, in the opinion ofthe pre-screening personnel, no scoring value. Thus no scoring data wasstored for the first 20 frames Then finally, in the 21^(st) frame ascoring opportunity was observed and coded. Here is what the 21^(st)frame looks like. The visible background on the screen is devoid of anyaction elements. At right foreground is an elevated authority figuregazing directly down at the viewer and explaining something intensely.At the opposite and lower corner of the screen is a motionless goldenretriever looking approximately in the direction of the viewer. Withinthe story of the vignette, apprehension of the vivid and communicativeexpression on the speaker's face is necessary to understand the contextof the vignette.

Looking at the database tables (FIG. 7 is one sample embodiment of thedatabase table structure), there are no pre-scored reviewer controlsstored for the first 20 frames. In one data strategy there are 20records in the primary database (associated with the first 20 frames)with no child records because there are no scoring values to be storedin said child database table. (The video reference field may optionallybe used to sync with the appropriate video frame, Alternatively, thecontrol software may simply start the first primary record with the21^(st) frame number. In that somewhat more efficient data framework,the control software reads a primary database record and, if it is for aframe yet in the future, simply waits until the frame whose number is inthe frame number field comes up. Still other implementers will implementcontrol software that will not directly command the execution of eachframe but will otherwise synchronize eye-tracking, scoring, and videoexecution. In fact it has even been found practical to simply identifythe approximate instant frame number being executed based on the timeelapsed since video initiation using the system clock of the high-speedpersonal computer that runs the control software. However, in theexample file structure of FIG. 7, there is one primary database recordfor each frame and this primary database record is indexed (in aone-to-many data relationship) to point to any number of child recordscontaining scoring information for any number of scoring elements forthat same frame. There is also a video reference field in the primarydatabase which may be used for matching video image data and/orproviding data supportive of controlling its execution.

Now, as the 21^(st) frame is being displayed, the control softwareinterrogates the 21^(st) record in the primary database and the firstmatching (related) child database record as illustrated in FIG. 7A. Thevalue of the vignette number field is the integer “1” indicating thatthis is beginning of the first of vignette's scoring (no previous recordhad a vignette number). The meaning attribute field has an “11” whicharbitrarily, for this example, identifies “interpersonal eye engagement,opposite sex and adult”. It is not necessary for the control software toperform any special functions based on this meaning attribute value(there are plenty of control elements in the child database that thecontrol software can use to grade the viewer). However, implementerswill inevitably add special grading elements for certain meaningattributes and this also provides a handle to facilitate that.

The target coordinates in this example (00743,00216) are the location ofthe left eye 802 of a right-facing adult near the top right hand cornerof the screen 801 as schematically illustrated in FIG. 8. (For the sakeof space in FIG. 8, the number of pixels on the screen schematicallyillustrated does not necessarily match the pixel density or aspect ratioof many screens.) Using location attributes beginning from the top lefthand corner of the screen, the target coordinates are 743 pixels to theright and 216 down. In FIG. 8 this can be seen as the intersection ofthe two dotted lines 806 and 807 which are illustrated to cross thosecoordinates.

The target strike value is identified as −41. Here, in this example, thepre-screening personnel appear to have assigned a negative value for thecondition when the viewer precisely engages the eyes of an elevatedadult member of the opposite sex ostensibly presuming that it is notindicative of a positive diagnosis for autism in the scene of theinstant frame. Screening personnel will, of course, establish their ownscales based on experience, changing understandings, and computeranalysis-generated values that will be discussed later.

The target full-value radius field has a value of 27 pixels in the datafield shown in FIG. 7A. It can be seen as a circle 803 of radius 27surrounding the target coordinates. Let's presume that the viewer's POI,805 in FIG. 8, was at (705, 261) as drawn (805) and thus fell outsidethat target full value radius 803. Thus, since the POI missed the circle803, the score is not affected by the −41 target strike value—yet.

However, prescreeners may assign a lesser value (a Depreciating RangeScore) for nearby areas. Only one example of the many applicableapproaches for this is illustrated based on the spatially depreciatingresponse curve field in the child database whose value is “3” in theexample shown in FIG. 7A. That number simply identifies an algorithmchosen by implementers and known to (i.e., operable by) the controlsoftware. The control software uses a logarithm-based algorithm,identified as 3 in this simple example, to depreciate the score overrange of about 75 more pixels based on the purely exemplary equations:

N=((ABS(X _(POI))−ABS(X _(TARGET)))̂2+(ABS(Y _(POI))−ABS(Y_(TARGET)))̂2)̂0.5−TFVR

DepreciatingRangeScore=target strike value*Ln(2*(SDRR−N))/5

where Ln is the natural logarithm, X_(POI) and Y_(POI) are thecoordinates of the instant POT, X_(TARGET) and Y_(TARGET) are the targetcoordinates, TFVR is the target full value radius, and SDRR is theSpatially Depreciating Response Range. Typically, this will only beexecuted when the instant POI falls outside the range of the target fullvalue radius during the display of this example frame (since a hitwithin the target full value radius gets full value). The value N is thedistance in pixels between the circle for target full value radius, 803,at the POI.

Thus, in this example, full credit is given within the target full valueradius and immediately beyond that radius begins to drop off with anexemplary function here which ceases to give any credit at all afterabout 75 pixels (the SDRR in this example) from the full value radiuscircle (the outer periphery of the scaled credit is illustrated by theCircle 804 whose radius is 102 since the target full value radius of 27plus the 75 pixel range of the spatially depreciating response curveequals 102 as the radius of 804 is so indicated). In this particular POIexample, N=32 (pixels are rounded) and the target strike value is thusmultiplied by about 89% thus the score is about 89% of −41 or −36.5(which, if by itself, would be contraindicative of the condition).Numerous other scoring elements and values will be used, these aresimple, examples of approach only. Other algorithms will, of course, userectangles and other area parameters.

The target time factor will be a factor in subsequent frames.

The score thresholds were set to 2 indicating that time-based scoringwill only occur when the other score for the instant meaning attributealready exceeds 2. This will be used in the next frame.

Had Skip frames been a positive integer, these factors would also beapplied in that many subsequent frames. Since Skip frames is entered aszero in this example, these exact criteria will not be applied to thenext frame but will require an additional database record (or many ofthem) if the implementer wants to continue this meaning attribute inthis vignette to (or beyond) the next frame.

For the 22^(nd) frame, there are two child database records indicated inFIGS. 7B and 7C. The first new child record finishes the vignette #1that we just began scoring in the 21^(st) frame (extremely shortvignettes are considered here for the sake of brevity) and the secondadditional child record begins a second vignette. Both vignettes areactively scored for this second frame. Of course at the implementer'soption, the second of these child records, FIG. 7C, for Frame 22, couldalso have been scored in the same (first) vignette. Even in thisabbreviated example, there are two meaning attributes beingsimultaneously scored in the 22^(nd) frame and there can also be many.Thus, any number of meaning attributes can be scored simultaneously inany frame, over any number of frames, and as part of mixed vignettes.The impacts of this multi-dimensional scoring approach will be seen tobe especially useful when using the Monte Carlo optimization.

Thus, in 7B, the only field that has changed is the frame number and aslight change in the target coordinates because of slight movement ofthe target, here the left eye of a right facing person (right eyehidden), between frames. The time factor is 5 which, in this example, issimply the number of an implementer-chosen formula for valuing thelength of the period of fixation. Thus, if the score for this meaningattribute has been above the magnitude of the score thresholds (shown as2 in this example) in a plurality of contiguous frames, a bonus to thescore based on the implementer-chosen time factor formula is added tothe score for this meaning attribute. Thus, this is one means of valuinglonger fixations during periods where that persistence of fixation isindicative of a tendency towards the meaning attribute.

The score for vignette 1 will be totaled when the control softwarerecognizes the absence of meaning attribute 11 in the child table for asubsequent frame. In this short example, we will let the PrimaryDatabase Table for the 23^(rd) frame have no related child databaserecords for meaning attribute 11 and thus the scoring for meaningattribute 11 will be closed with whatever score had accumulated up toand including the previous (here the 22^(nd)) frame. In the scoring forthe 22^(nd) frame, because the score exceeded the score thresholdmagnitude (here a negative one, −2), any implementer-chosen creditalgorithm for time factor 5 will be applied.

In FIG. 7C we see the second child record related to the PrimaryDatabase Table having FrameNumber=22. Thus, a second vignette, #2, willalso be scored for this frame. Briefly, the target coordinates indicatefor vignette number 2 in this frame a full credit for when the POIoccurs at 807 in FIG. 8 or at least within the target full value radiusof 125 shown by circle 808. If the POI misses this but falls within theSDDR circle, 809, a partial credit will occur. These credits will bepositive based on the target strike value of +38 since the implementerin this example assessed this area to be indicative of where a personwith the condition being tested might look.

Again, any number of areas may be scored for each frame. In fact, everypixel on the careen may have a positive or negative value associatedwith the POI occurring there. These scores can be summed by meaningattribute, vignette, and full session. Each time the control softwarescores a frame and finds that a previously executing vignette has ended(since it is not represented in this frame), the scoring for thevignette may be totaled and cross tabulated in any number ofcombinations with other values per implementer preference.

It would appear that this explanation of only one of many datastructures and control program operations applicable to the currentinvention is excessively extensive. Indeed, many will use, alternativelyor additively, rectangles or other complex shapes to identify areas andrange values from a point or points within them, often causing thetarget coordinate to be the “center of gravity” of even very irregularsuch shapes on the screen. However, this excessively extensivedescription of example fields and their uses will be further usefullyapplied to explaining both the optimization process and non-pre-screenedembodiments further below.

E. Video Display Real-Time Adjustment

In the simplest and preferred embodiment, the video display is nowadjusted, unless implementers prefer to use the extended responseelements in step H of FIG. 3 instead, responsive to the score for thecurrent frame. There are a number of practical and effective responsesknown to be effective for the remediation of certain conditions andother applicable ones will arise.

The mechanics, applicable to the current invention, of degrading thevideo image to motivate the viewer are too numerous to list. Anyequipment or process that can be directed to modify an image isapplicable. For example, in the simplest embodiment, an ordinary desktopcomputer with a hard drive, video card, screen, and software to degradethe image before it's displayed can fill the equipment-based roles ofprocessor, data source, video controller, display device, and imagedegrading element (listed respectively). The image degrading element canalso be a hardware device placed between the video controller and thedisplay where the processor is operatively connected to that hardwaredevice in order to adjust the amount of image degradation.

Defocus

Defocus has been shown to be a powerful stimulus for corrective eyefixation. Much of the brain's processing capacity is dedicated to thefull apprehension of captured visual images and brain plasticity hasbeen observed aiding in the improvement of dysfunctional eye-fixationconditions. Responsive to an adequately positive score, the image isblurred. In embodiments similar to the one shown in FIG. 1, there are anumber of ways known to those skilled in the art to do this. Perhaps thesimplest is to use software utilities (e.g., the Intel toolbox for videoprocessing) called by the running control software for a degree ofdefocus relative to the score. In worn embodiments similar to FIG. 5 andhaving electronic focus control, e.g., electro-optic lenses and spatiallight modulators (SLM), defocus can be applied to even very portableembodiments and optionally to embodiments requiring no pre-screening.

Color Deprivation

independently or in coordination with other image degrading

Localized Defocus

The area of defocus can also be confined to an image area implementersdesire to highlight or direct the viewer's attention away from. Forexample, the responsive real-time adjustment of step E in FIG. 3 for aan autistic viewer watching an area indicative of a positive score(meaning the presence of the malady), that area may be defocused. Inprogramming terms, using the program code snippets of FIGS. 11-13 andthe much discussion regarding them, this area to be avoided (and thuslocally defocused) may be described as the area “covered” by scoringanchors (e.g., from the file ScoringChild) having highly positiveTargetStrike field values). These anchors are also referred to herein astarget locations. The actual areas thus defocused are the areas centeredaround the TargetCoordinates location and surrounding area within aradius of the sum of the values for TargetFullValueRadius andSpatiallyDeprRespRange. The degree of defocus within theTargetFullValueRadius may optionally be higher (optionally based on thevalue of the TargetStrikeValue) than just beyond the circle with thatradius but still within the radius that includes theSpatiallyDeprRespRange (optionally based on the values from calculationof the SpatiallyDeprRespCurve).

Localized Focus

For example one desired remediation cue for autism benefits from anability to both limit vision to stimulate a change in POI and toindicate where the subject should be looking by the location. Thus, astimulation cue for strabismus to direct a viewer to look at a certainarea may be a general defocus of the image except for a localized areawhere implementers want the viewer to watch. Programmatically, thissharp focus in the preferred area of POI location can be effectedsimilarly to the Localized defocus discussed just above here except, ofcourse, that the area selected using TargetCoordinates fields, etc.,would be based on areas defined by scoring anchors having very negative(i.e. desirable) values for TargetStrikeValue (as opposed to thepositive values preferred in the previously explained defocus example).

Diplopia

Similarly, software-directed diplopia is a powerful incentive forcorrecting fixation and, for that matter, stopping whatever else it isyou are doing appears to be causing it.

Vignetting

Vignetting is a powerful option that not only spatially indicates (atthe center of the vignette area) either the current point of fixation orthe implementer suggested point of corrected fixation (depending onimplementer preference). Applicable to being used at a gradient, thedegree of correction required can be thus be proportional to theperipheral visual obstruction driven by the magnitude of the vignetting.As is discussed elsewhere herein, this can be especially useful inembodiments where the viewer's forward view is important forperspective, navigation, or spatial perception. The narrowed field ofview through the peripherally shaded view, whose effect to “aperture”and the darkness of the peripheral shading are responsive to the degreeof the condition to be treated. For a person with autism, for example,the center of the vignette can be over the suggested viewing areaforcing the viewer to look where he should in order to have good vision.In another example for a person with strabismus, the location of thesuddenly-appearing vignette's center tracks the position of the weak eyerather than the strong eye forcing the user to direct his strong eye inorder to have good vision.

Dimming or Blanking

Dimming or, perhaps for extremely highly positive scores, blanking theimage altogether also forces the viewer to correct undesirable visualfixation. In embodiments applied to strabismus, this dimming can bedirected to occur selectively in only the strong eye analogous toconventional treatments. In portable embodiments, this can beaccomplished with shelter glasses and for HUD's and interferenceprojected image has a similar effect as discussed elsewhere herein.

Targeted Direction

When a POI occurs in a highly positive scored area of the image, theimage display (whether it is a monitor, heads-up display or otherdisplay) simply displays an overlaid image over the normal image at alocation in the image that the implementer selected to be behaviorallydidactic. To use an earlier example, when the POI falls upon and area ofthe image normative to the condition being treated, a crosshair, circleor other indication over or around the preferred (negative) area leadsthe viewer to a behaviorally improved POI. In the preferred embodimentusing targeted direction, highly interruptive, blinking, spinning andotherwise attention drawing targets are used.

F. Record Cycle Data and Scores

The score is recorded enabling later cross tabulation of data overmultiple vignettes and combined meaning attributes.

G. Real-Time Transmission

Optionally, the data can now be transmitted wired or wirelessly eitherin real time or in batch mode to caregivers for analysis.

H. Extended Response Elements

As optional replacements for or augments to the above video displaystimuli, extended response elements are effective means forcommunicating both function specifics and magnitude of importance. Forexample, a given sound or even verbal audio message, produced by acommon voice simulation circuit and a small speaker in any of theembodiments considered herein, can indicate a specific recommendedresponse to underscore or explain a visible corrective stimulus. Theycan also be used to underscore the importance of the stimulus;particularly when the viewer has been inadequately responsive.

I. Next Frame

The keyboard or any other user interface element is then interrogated tosee if the viewer has indicated a desire to stop the video. If not, thenext frame is begun and the cycle repeats typically with step B in FIG.3.

J. Save and Optionally Transmit Final Results

If the viewer has indicated a desire to stop viewing, the user interfacereturns to the main menu.

Non-Pre-Screened Scoring:

Software for locating faces in an image and identifying the location ofeyes, noses, facial orientation, other body features, and even buildingand landscape features is well understood. It is also possible toestimate the distance to the person being viewed based on the distancebetween their eyes being recognized. This software is often a modularfeature for camera image acquisition software. For example, this imageelement recognition software module (IERS) is commonly used with cameraranging and focusing software modules. It identifies an area in theviewing field of view (FOV) where, for example, it recognizes andlocates the eyes of one or more people. The focusing portion of thecamera software, responsive to this data for such location(s), can setthe camera focus to the distance sensed for that location by the rangesensor. Adding an IERS module to the control software of the currentinvention is straightforward and understood by those skilled in the artand thus will not be extensively explained here. In embodiments of thecurrent invention including this optional element, the software caninterrogate and score, not unlike the scoring above, any video imageand, where desirable, do so in real time.

Positive Reinforcement

It would be unfortunate if the current invention only provided negativereinforcement. Of course, the sudden absence of any of the negativereinforcements listed above responsive to a subsiding of an adversebehavior is in itself an instant positive reinforcement. In thepreferred embodiment, all stimuli are responsive on a gradient. That is,the more adverse the behavior, the greater the magnitude of thecorrective stimuli. Similarly, the more healthy the behavior the morepositive it should be. In fact the breadth of that gradient in negativestimuli can be expanded with the addition of additional positivestimuli.

Many of the same attributes perceived as negative stimuli may be chosento be especially enhanced when the calculated scores are not only notadverse but are very healthy. For example, in the presence of veryhealthy behavior, image color may be especially enhanced beyond aslightly dull default value, surround sound can replace a singlespeaker, music can accompany an otherwise quiet video, and the size ofthe image on the screen can be increased from the smaller default valueto full screen.

Seated Real-Time Applications not Requiring Pre-Scoring

In an embodiment like FIG. 1, the scoring software hosted by thecomputer 109 performs the IERS modular function on each frame of videobeing displayed on the screen 102 and reports recognition locations andidentifications to the scoring module) while recognizing the POI throughthe eye-tracking module 108.

The scoring software, thus advised of a recognized pattern's locationand identification, scores based upon implementer-provided criteria asis detailed herein.

For example, consider a single table (similar to or identical to thechild database discussed above except without frame number, vignettenumber, and skip frames and indexed by meaning attribute. Here, forexample, we consider a meaning attribute number 107 arbitrarily chosenhere to score, where applicable, a POI at or near a recognized pair ofeyes. Then, in real time, the IERS locates a set of eyes in the cameraimage being viewed and passes the displayed image coordinates of acentral location representative of that location to the control softwareat the same time that the eye-tracking module provides the coordinatesof the location of the POI. The scoring module of the control softwarethen looks for a record with a meaning attribute field value equaling107. In one example embodiment, the implementer has chosen to provide ahigher target strike value for eyes recognized and located above thecenter of the screen analogous to looking up at someone. Thus, if one ofthe records with a 107 in the meaning attribute field has a targetcoordinate TargetFullValueRadius (or width and height for embodimentsincluding the rectangular area identification) inclusive of the instantPOI the scoring feels that record can be used to score the instance asdescribed herein. Responsive corrective stimulation can, then, beeffected in real time.

It should be noted that scoring does not need to use a database approachbut, as is obvious to those who write software, can be hardcoded torecognize and score a POI in any area of interest with asoftware-designated value. Also, distance to subject may also be afactor considered for scoring. For example, a person fearful of orreticent at eye engagement may find it even more difficult to achievethat engagement close-up. Thus, where either rangefinding sensors areprovided and operatively connected to the control software or whereranges are estimated based on inner-papillary distance (orinter-eye-socket-center distance or other means) scoring criteria willbe amplified (normally multiplied by a conversion factor greater thanone) to be higher for nearer distances than for far distances and that,in the preferred embodiment, along a gradient responsive to thedistance. Also, where attention is part of the score basis, when a POIoccurs far beyond the location of the image, that may also be included(scored) towards a more positive score.

Even in this non-pre-screened application of the current invention,remediative action may be accomplished in real-time responsive to theviewer's POI. Thus, in response to a highly positive score (magnitudeschosen by implementer), any of the remediative responses describedherein for step E of FIG. 3 can be applied.

Worn Real-Time Applications not Requiring Pre-Screening

In one embodiment a viewer-worn camera (e.g., a worn hardware assemblyincluding a forward camera like 504 in FIG. 5 and an eye tracking camera502 or alternative embodiments not unlike the Google glass forward viewcameras worn like or mounted to glasses) is operatively connected (bywire or wireless communication) to a scoring computer hosting thescoring software. In the preferred embodiment, all of the components areminiaturized and as is broadly available.

Applicable to both autism and strabismus, the playing video in a worn,real-time embodiment, is simply replaced by the live image feed of theforward camera. However, these live cameras also have frame rates andthese, or any other time-based delimiters, serve the same function asthe frames in the other examples. In the preferred embodiment, theforward view camera image comes from a worn camera like FIG. 5 and,along with the eye tracking camera data from 502, is connectedwirelessly or wired to a worn processor (not shown). The softwarerunning on the processor recognizes patterns (e.g., eyes with noses andhands using IERS) and, based on the POI's proximity to a central pointin that recognized pattern, scores a positive or negative result asdiscussed herein based on an implementer choices and the condition beingdiagnosed and/or treated.

Whether the application is for strabismus, autism, both, or for otherapplications, all such embodiments will provide, responsive to apositive score, (even though that positive score is calculateddifferently for different applications as discussed herein) a correctivestimulus. The magnitude of that stimulus is responsive to the magnitudeof the positive score, normatively affecting the ability to see theforward view but sometimes additively or alternatively including otherstimuli like sounds.

For example, when the application is for strabismus there are a numberof real time obtainable indicators for both the presence of binocularinfidelity and its magnitude. For example, when one eye moves and theother eye does not, when there are differences in eye elevation asopposed to the two vision axes at least essentially sharing a transverseplane, all of which are easily obtained from eye tracking software insome of which are already provided in that form by the eye-trackingsoftware.

When the eyes move in different directions e.g., one to the left and theother to the right Eye elevation, as discussed above, is an indicationof binocular infidelity.

Also, esotropic strabismus is easily recognized by the exceptionallyclose intersection of the vision axes as reported to the controlsoftware by typical eye-tracking software (alternatively reported asdistance to vergence). Exotropic strabismus is similarly recognizablewhen excessive distance to target calculations occur are received fromthe eye-tracking assembly.

Inactive/active partners: When one eye moves and the other does not,this is an indication of dysfunction.

Responsive to a positive instant score (indicating the presence of thecondition to be treated), the wearer is equipment-stimulated to adaptthe behavior. If the worn assembly includes electronically focusedand/or image-shifting lenses (e.g. electro-optic), defocus or diplopiaas described elsewhere herein can be used. In this case, the controlsoftware, operatively connected to the electro-optic controls for theelectro-optic lenses, instructs the electro-optic controls to shift thefocus along a gradient responsive to the magnitude of the positivescore.

Where the worn assembly includes shutter glasses, the image may bedimmed or blanked responsive to the magnitude of the positive score asis commonly understood and practiced by devices implementers whosedevices communicate with shutter glasses to determine their periods oftransmission and periods of closure. However, in any embodiment wherethe user needs to navigate (particularly with such a potentiallyportable embodiment where the viewer walks around unrestricted),vignetting or targeted direction are preferred corrective stimuli.

A related embodiment involves any form of heads-up display (HUD). In thepreferred embodiment, the HUD places the displayed image over a forwardview. One example of this is reverse projection allowing a projectedimage to be seen “over” the view through the glass. As above, theforward camera captures the forward view and the eye-tracking camera,similar to 502 in FIG. 5, and software locate the POI. The portableprocessor hosting the scoring software and connected by wire orwirelessly to both cameras and to the worn imaging display places atargeted direction symbol/icon or location-indicating vignetting overeither the instant POI (which best allows the wearer to see the forwardview) or over the suggested point of fixation (determined by implementerpreference and application).

However, in the preferred embodiment of a worn assembly intended forportability (where the processing computer is also miniaturized andworn) remediation responses that favor continued and safe navigation areused. For example, one response is a blurring (defocus) of the imageresponsive to the magnitude of a positive score. But when the viewer is,for example, walking, central vignetting (permitting a gradientlyapplied and slightly narrowed FOV around the instant POI) or targeteddirection overlays allow the viewer to see where he is going while stillrecognizing a compellingly visible negative response. Other alternativesinclude sounds and other stimuli.

Non-Subjective Multi-Factorial Optimization: Applicable to any Diseasew/ Significant Effects on Eye Vergence, POI Selection, or Persistence ofFixation

The ostensibly excessive description of data fields and scoring detailsabove was thus detailed to also provide breadth in the reduction topractice of target capture in an image, the process of valuing it, andscoring the effects of a plurality of simultaneous points of interest aswell as to facilitate the explanation of both optimization proceduresand scoring for alternative embodiments.

With that data structure in mind and an understanding thus accomplishedof the effect of field values, we now consider another layer of functionand process associated with both recognition of new and unknown(typically less obvious) condition signatures and the optimization of asystem for applying them to more reliable diagnoses and more effectiveremediation.

The current process for identifying dysfunction signatures in themassive amounts of data from clinical tests tends to be both random andempirical. The conventional process for quantifying the importance andvalue of dysfunction signatures as well as their multivariate impact onother related signatures is even less scientific and even moresubjective. Eventually, with much experience, clinical researchers willobserve what is essentially only the tip of a much larger iceberg inonly the most obvious of signatures. The relative and comparable weightto be appropriated for each of the potential legion of such signatures(many of which are still beneath the scientific radar) is extremelydifficult if not impossible to apprehend. Grasping, much less applying,the interrelation of these many signatures, including their constructiveand destructive interference in augmenting or attenuating each other'sobservational significance is beyond the pale of even the most diligentof researchers.

However, the data capture assemblies, predictive stimulus,complexity-independent data structure (able to deal with any number ofphysiologically revealing attributes and their signatures simultaneouslyin a single frame or other increment of time), and the plurality ofsimultaneous scoring methodologies of the current invention togetherprovide the foundation for a second layer device and process forautomatically converting clinically captured data into statisticallyverifiable, unbiased, and non-subjective scoring tables. These new,novel, and clinically-derived scoring parameter tables (SPT's) of thecurrent invention in coordination with the scoring methodologies of thecurrent invention thus enable a process for developing the firstscientific device and process for statistically verifiable and unbiasedidentification of even a large number of interacting and potentiallymutually interfering physiological signatures. These can be applicableto any malady significantly affecting orthophoria, normative vergenceposition acquisition, and/or qualitative target selection (selection ofa POI based on a conscious or subconscious preference for the identity,nature, or location of the target).

Clinical-Capture for SPT Development; an Example:

Consider first a video that has not been pre-screened (and will not needto be prescreened by a human). However, using processes and procedureswell known to those skilled in the art, the video is displayed on any ofthe display mechanisms discussed while the viewer watches and the POIlocations are captured on a frame by frame basis. In the embodiment thatis simplest to explain, the assembly of FIG. 1 is used. Fromwell-established and long-term clinical evaluation, a researchpopulation is selected, including known healthy and known conditionpositive subjects, and each clinical research subject is hierarchicallyrated relative to the magnitude of their condition. For simplicity inthis explanation, however, we will simply break the research populationinto four groups based on subject condition (here in increasinglypositive order): 1) healthy, 2) high-functioning positive, 3) positive,and 4) severe. However, more levels will often be used. (These overlygeneral terms are used because there are a number of differentconditions applicable to this use of the current invention.)

Step One: Subject Data Capture:

While each subject watches a series of preferably contiguous videovignettes, their POI's on the screen are captured and associated withthe instant frame. In an exemplary database for storing this captureddata, the field structure can be as lean as illustrated in FIG. 7D.

Many implementers will prefer and use arrays over the databasestructures used herein but the database structures are an easier way toexplain the process.

-   -   I. Step one Parent table: (indexed by subject condition)    -   II. subject number (Integer)    -   III. subject condition (Integer) (e.g., 1 for healthy)    -   IV. Child table: (indexed by subject number+frame number)    -   V. subject number (Integer)    -   VI. frame number (Integer) and    -   VII. target coordinates (String, 10)

The addition of vignette number may be desirable for some implementer'sto aid in the later process identifying the meaning attributesassociated with points in each vignette but this is not a necessity. Foreach testing subject a parent table record is created and for each videoframe watched by the subject a child record is created. These tableswith the indexes shown are especially useful for managing step one dataparticularly during the early data capture process which can require andextended period of time. It can optionally also be used to supportadditional display software that allows clinicians during step one toview the video and see the subject's POI's indicated over the videoimage with crosshairs, etc. in real time. However, a separate set ofdatabase relations or structured query language (SQL) calls orequivalent methods may be preferred for step two.

Step Two: Pattern Convergence Recognition (PCR)

There are a number of optional and applicable methods for accomplishingPCR and at least two of them will be described as examples herein. Thefirst approach (Approach A) is extremely didactic for an understandingof the logic and principles behind the second approach (Approach B).Similarly, the programmatic elements later described for Approach B aredidactic in understanding the practical programming for softwareautomation of Approach A. The automated results of both Approach A and Bcan be used for later scoring of new subjects or may be optionallyoptimized, as will be described below, for improved results.

Approach A for PCR:

In the second step for SPT development, which typically occurs after allof the step one capture is complete, the subjects have gone home, and wenow prefer the above data in different relationships supportive of PCR.This can be accomplished through additional indexing relations,conversion to a different database structure, an SQL selection or otherequivalent step(s), as will be understood by those familiar databaseprogramming, accomplishing the same goal of the following data structurepreferred for this first example of the PCR process:

SubjectCondition

FrameNumber

POICoordinates

sorted, selected, or indexed in order of SubjectCondition+FrameNumber.

A less lean database structure may also be used to provide betterbacktracking ability and is described in this example.

Then, step two PCR software interrogates each record having subjectcondition equal one (since these are in order ofFrameNumber+SubjectCondition and we start at the top of the file)starting with frame number one and SubjectCondition=1 to startconsidering some POI's from healthy viewers. Recall that for eachsingular frame number there will be as many records with POI (targetcoordinates) data as there are subjects with subject condition=1 (andthe same for the other condition levels). Thus, for each of thepotentially numerous records with subject condition=1 and frame number=1there is a POICoordinates field value indicative of one subject's POI onthat singular frame.

In an optional researcher-directed next step, a density map, understoodby statisticians, may be created. This may be thought of as a dotplotted at the target coordinate location for each such record havingSubjectCondition=1 and FrameNumber=1 on a Cartesian map like 903 in FIG.9A conveniently having X and Y coordinates equivalent to the number oflateral and vertical pixels respectively on the screen and beginning atthe top left corner for 1,1. Thus, a record having target coordinates0074300216 would be plotted as a dot at the location indicated by 901(leading to the center of the concentric circles) in FIG. 9a . The abovereferenced records, all for POI's for healthy people in frame one, canbe seen as plotted in the dots like 901 (located at 743, 216) and 904 inFIG. 9A.

The concentric circles like 804 and 803 in FIG. 9A together serve as ascoring factor or “anchor” (because they define a fixed and visiblecontainer securing/containing the POI dots). Thus references to anchorsherein point to scoring elements in a scoring template like 803 and 804in FIG. 9A and to their representation in scoring databases (e.g.,ScoringChild in FIG. 11).

Because eye-tracking equipment and displays vary by resolution and indifferent dimensions, implementers will, where any differences in x andy pixel density is sufficient to create any significant accuracy issues,need to adjust the values so that those values are equivalent as isunderstood by imaging personnel,

Location and Calculation of Target Full Value Radii and SpatiallyDepreciating Response Curves (Also Referred to as a Declining PowerFunction which is a Nomenclature that is Inclusive of Both CurvedDepreciating Responses and Those that are Linear):

The target full value radius for healthy people for this frame (whichmay later be used with or without further optimization in scoringrecognition of negative traits for the condition) may now be identifiedusing statistical methods for density pattern grouping influenced byindividual implementer preferences. For example, areas of density in themost concentrated of areas can be identified as illustrated by thecircle 803 in FIG. 9A (or the rectangular, elliptical, and otherwiseshaped equivalents thereof). The radius of this circle will vary byimplementer preferences and, for example, can be based on the radiusrequired for the circle 803 to encompass one standard deviation (of thenumber of POI's for the condition level) divided by the number ofidentified target areas. For example, in FIG. 9A, the clustering of thedots indicates that there is only one target area almost completelyencompassed by the small circle 803 in the larger circle 804. Thus, forN qualifying records (indicating, in this example, that they were Nhealthy subjects), the radius of the circle 803 may be chosen such thatthe circle 803 encompasses Q dots where:

Q=σ/T

where σ is the standard deviation presuming a Gaussian distribution andT is the number of targeted areas like 803 (of which we have only one inthis illustration). Thus, if N=100, 803 has been thus selected by thecontainment of 69 dots (FIG. 9A is for general use and does not have 69dots in 803.) This usable, yet pre-optimization, radius value may now beused as the target full value radius for the first meaning attribute offrame one.

Similarly, and again as only an exemplary calculation, the radius of thelarger circle 804 associated with the spatially depreciating responsecurve, may be driven by the FWHM of the distribution divided by T or, inthis example of N=100 and T=1, i.e. barely big enough to contain about76 of the dots.

Whatever values are chosen by implementers for defining these areas ofcontainment, they may also be further adjusted an improved in thesubstantial optimization process that optionally follows.

Calculation of Target Strike Value:

All of this process is repeated for each of the groups (in thisexample: 1) healthy, 2) high-functioning positive, 3) positive, and 4)severe) to obtain different results with different scoring values basedon the known groups. Although the target strike values for the healthygroup will typically be negative, the values for the other groups willtypically be positive for the meaning attributes associated with thecontainment areas thus defined. Further, the numerical magnitude of thetarget strike value assigned is responsive to the level of the group.This is one reason why more than these four sample areas will often beused. For example, 10 levels of stratification from negative to highlypositive will logically be associated with 10 magnitudes of targetstrike value thus first defining the target strike values by the levelof the group. Within the level of each group, implementers may alsoadjust the target strike value by the density towards the centers ofconfinement areas. In the example above where the target full valueradius was driven by the standard deviation, the larger the circle 803has to be to confine those first approximately 69 dots, and similarlythe larger the circle 804 has to be to contain the FWHM, the lesstightly clustered the results are. Thus, for the healthy group (#1) andwhere those circles center around a set of eyes looking at the viewerand where the condition being analyzed regards autism, a less tightlyclustered result may result in a lower magnitude negative score. Whilethere is still some subjectivity on the front end, albeit by thoseskilled in the field, the optimization process that optionally followsis designed to remove subjectivity.

Recognizing and Responding to Many and/or Large Confinement Areas:

Scenes will typically be selected by implementers to encourage asingular area of fixation in a relatively small area like 804 determinedby negative (healthy) subjects and a singular area of fixationdetermined by positive subjects both being responsive to action orevocative key scene elements. Nonetheless, some groups of frames willunavoidably allow idle time resulting in more than one area of primaryfocus for known healthy subjects and one area of primary focus forsubjects known to have the condition. When that is the case, the dotpatterns will not be tight but, instead, spread out resulting inspatially large confinement areas. This is numerically and visiblyobservable as a large target full value radius, here driven by standarddeviation, illustrated spatially here as 803 and/or a larger 804 drivenby the particular spatially depreciating response curve algorithm used,which in this example based 804's size on FWHM. Where these spatiallydriven numerical values are very large, there may be much less meaningor data reliability associated with them so implementers can simplyrespond by providing no scoring records for those frames thus makingthose frames irrelevant to the score.

Also, and separately, there will, of course, also be many sceneconditions where it is simply normative for any given subject to look atmore than one confinement area and yet there is significance to each ofthat plurality of confinement areas. When that is the case, thoseconfinement areas will be more reasonable in spatial size (smaller thanthe meaninglessly large ones just discussed and dismissed) and therewill be a plurality of them. For example, there may be multiple thingsin a larger general area that a healthy person may look at and it mayvary which they look at first. Similarly there may be multiple thingsthat persons at various levels of the condition will look at. Thus,because these scene elements have meanings, a cluster of POI responses(seen in FIG. 9A as dots) will be tighter (more densely packed than themeaningless frames just discussed that will normatively be discarded).In these cases, it is not only unavoidable but beneficial to havemultiple confinement areas for scoring even if they overlap as will beseen.

It should be repeated that, despite the examples given, the values forthe target full value radius and the algorithms in any selectedspatially depreciating response curve can be implementer-selected by anymeans including prior implementer experience (with subjectivity dealtwith in subsequent optional optimization). In examples like the one justdiscussed where the single negative target full value radius was drivenby the standard deviation of the full distribution (for example, whenperforming this process on the first group, i.e. healthy subjects, thefull distribution is all POI's captured from healthy subjects for thatframe which should be the number of healthy subjects, 100 in thisexample), it was appropriate to divide the portion of that 100associated with a Gaussian distribution by one since there was only onenegative confinement area (with an 804). This is why, in the aboveexample calculations, we set T to equal one.

However, when there are multiple negative or multiple positive clustersof significant (e.g., statistically dense enough to represent a tendencyrather than random action) POI's (dots), implementers might presume thateach of the individual confinement areas like 804 would representGaussian distributions of subsets of the total population. This is oneof the reasons that 100 (used in an example here as the total populationof healthy subjects solely because it is easier to explain withpercentages) is a placeholder for an ideally larger number. If the Tconfinement areas share essentially equal POI hits for the instantframe, the above logic of dividing by T is useful. Otherwise,implementers can apply weighting as appropriate.

This implementer-directed process is intuitively satisfying, favorsapprehension of error, and will be preferred by implementers preferringa hands-on, didactic process particularly while learning the nature ofresponses to a given video and/or for a given group of subjects withknown conditions. The steps of automation of many or all of these keyprocesses is understood by those skilled in programming, statistics,graphics, and imaging. Further, Approach B, described below, providessubstantial additional and applicable programming information applicableto any desired automation of this Approach A.

Approach B for PCR:

There can be many frames in a video resulting in much work in PCR and,with that tedious work, opportunities for error. An alternative approachto performing step two of PCR (including anchor finding), described now,can be less subjective and more automated. Towards this end, there arestatistical packages available whose extensive functions identifyclusters of points and the statistical nature of those clusters alongwith other features applicable to the PCR process of the currentinvention. However, while applicable to the current invention, they tendto be too long in both programmatic code and in functional descriptionto be easily included in the text of this specification. While theseapplicable packages may be used to fulfill at least some of the PCRprocess, a brief alternative example approach will now be summarized. Itwill be best understood when recalling that our goal is the same goal wehad for Approach A, that is to identify and value the clusters of POI's(that is, the anchors seen both in the circles of FIG. 9A and the datain database files like ScoringChild) for each frame. Like Approach A,once the areas are identified and at least approximately valued, theymay be optionally and subsequently optimized.

As in the example description for Approach A above, we will sometimesdescribe performing PCR on only one group (e.g., healthy subjectssuggesting negative scores) realizing that the described operations mustalso be performed on the other groups in order to have a scoring systemrepresentative of them. However, although not the preferred embodiment,PCR may be performed on any individual group to identify and valueconfinement areas and later be used on the general population to atleast screen for the characteristics of that potentially singular groupor of a smaller subset of all groups. Thus, a minimized embodiment ofthe current invention might gather data from only one condition groupand then use the magnitude of scores based only on anchors for thatcondition as an indication of the instant reviewer's degree ofsymptomatic affiliation with that group.

Location and Calculation of Target Full Value Radii and SpatiallyDepreciating Response Curves:

Let's consider FIG. 9B without benefit of the circles 803, 804, 808, and809. In other words, we see the dots but we haven't yet figured outwhere the confined areas should be located based on cluster scatteringand there are no humans to look at a scatter map and visibly recognizethe centers of or the separations between such confinement areas

There are many applicable approaches to identifying the centers andlocations of clusters including the application of well-knownstatistical principles including calculating root mean squares of POIlocations that are progressively further from a point in the image beingconsidered. There are many such approaches that will certainly be usedas part of the current invention by implementers. To describe apreferred embodiment, an alternative and simple, yet tunable, approachwill be described here.

To communicate somewhat complex ideas, we will use herein, in additionto textual descriptions, program code snippets that are not necessarilyspecific to any particular programming language and are not intended tobe operable in their current form in any particular language or underany particular operating system but will be easily understood from thelevel of detail provided by most anyone skilled in the field of computerprogramming.

Recall that we come here with all of the POI data in the data structureof FIG. 7D which is useful for the work associated with the capture andmaintenance of captured data. However, in the data that will be used inthis step, subject number is not needed or desired both for dataefficiency and to enable maximum (and often Independent Review Board,IRB, required) separation between subject private information andpotentially publishable clinical data. Thus, we now produce a singletable from the table of FIG. 7D containing, for every POI captured, onlyframe number, subject condition, and target coordinates indexed by framenumber+subject condition. This table can be stored in any format andreused. For speed of processing, these data may be placed in theone-dimensional arrays:

1. frame ( ), condition( ), and POI( ).

where the suffixes of each of these arrays recall the order of thetarget coordinates and the associated frame and condition data. Thus, ifthere were 100 healthy subjects (with subject condition #1) the valuestored for frame(1)=1, condition (1)=1, and POI(1) is equal to the POIlocation of a healthy subject when watching frame number 1. The valuestored in frame (101) would still be=1 (we're still in the first frame)but the value for condition (101) would be=2 and the value for target(101) is the POI of a subject watching frame number 1, etc. Thus, thearray is ideally placed in an order that can be scanned with anincrementing suffix to rapidly consider, through each frame number andsubject condition, all of the POI's that occurred in a frame forsubjects of that condition. Exemplary use of this preferred arraystructure is in the programmatic descriptions of FIGS. 11-13. Where IRBrequired, the initial capture of this data may be put directly into thisarray format in the first place maintaining the desired order butremoving any subject information reference.

Example Token Code:

The general and partial code segments shown in FIG. 11 are merelyillustrative of the basic approach. They also use names longer than manynaming conventions actually permit to better convey more clearly whatvalues are. Further below many of the programmatic steps will bedescribed in a narrative. These programs pick up in FIG. 11 after thecaptured POI data whose capture was described above is transferred tothe first set of global arrays described above (condition( ), frame( ),and POI( ).

No particular screen resolution is inherent in the implementeralready-established variables for)(Range and YRange indicative of thepixels of rows and columns respectively in the implementer's screenhardware. Also, some high resolution screens may provide more spatialresolution than eye tracking equipment may be able to match andprocessing speed slows when extremely dense matrices of many pixels areused. Where that is the case, implementers may choose applicableembodiments where every pixel location will not be considered as theyare in the code snippets we discuss here. For example, for a very highresolution giant screen somewhat distant from the viewer, it would notbe unreasonable for implementers to use, for example, only every 10^(th)or 15^(th) pixel location laterally and vertically in a sparser matrixas is easily performed by those skilled in the art and then matcheye-tracking captured locations to the nearest available thus-selectedpoints on those sparser matrices. In many applications this can be doneto substantially accelerate processing and minimize data storageoverhead. It will also substantially increase optimization calculation.

The steps already described above can provide usable data without needof further optimization. However, the structure and process of thecurrent invention allow it to thrive on even very large populationsamples of subjects enabling more accuracy and, additionally, apotential for discovering condition signatures in the data that would beoverlooked by humans awash in a flood of seemingly random artifacts

Also, optimizations may optionally be at the frame level. That is, eachframe can be optimized by all of the POI data for all subjects of allknown conditions viewing the video. This is, in fact, the first optionexplained in FIG. 11.

Alternatively, a singular set of optimization criteria can be obtainedwith automated optimization and used for all the frames of an entirevideo or multiple videos.

Very Brief Narrative of the Code Snippets:

A very brief overview of the provided code segments beginning with FIG.11 is now provided and will be followed by more detailed discussion ofkey points.

FIG. 11 begins with a typical setting of global (or public) variablesand default values.

Then the as yet empty database ScoringChild is opened and indexed as thefuture container for scoring anchors soon to be identified. The othertable, opened next, is for the later storage of SPT anchor scoringparameters and a measure (the field “Score”) of how well thoseparameters resulted in the recognition of a particular condition leveland, additionally, how well these condition levels were distinguishedfrom each other (i.e., how well POI scores from a person of onecondition was kept from affecting the scores for another condition thushelping to prevent false positives).

Next, the program GrandTour( ) is called which begins in FIG. 12A. Itbegins by iteratively testing a very wide range of parameters that areused in the identification, location, and valuing of scoring anchors(e.g., DefRange1, DefRange2, DefRange3, and MinHits and potentially manyothers that are not illustrated in this already not-so-brief example).

For each of the iteratively attempted combinations of these calculationfactors, the program MakeAnchors is called (seen in the lower half ofFIG. 12A and completed in FIG. 12B). Using the instant set of theseparameters to be tested, MakeAnchors creates a complete set of scoringanchors applicable to scoring any later viewer being tested. It is alsoselective in that it does not create anchor records for areas of littlePOI concentration. This program also answers the question “what kind ofcode can be used to turn raw subject data into scoring anchor templates”since that is precisely what it does.

However, this is the grand tour of all the possible iterations of thevery parameters that determine the size of the areas determined to beindicative of a condition, how their density or power is to bedistributed spatially, the valuation of a POI location at any point, andwhich image locations should simply be ignored when a POI is there.Thus, the purpose of MakeAnchors in this unusual application is to makeall the anchors for one iteration set of parameters and then score allthe POI data we have on all subjects of known conditions with thesecustom created anchors and then determine if that is good data. If it isgood data, then these are good parameters.

To do this, as soon as MakeAnchors returns to the deeply nestediterative loops at the top of FIG. 12A, the program ScoreVictims iscalled. Some might call this program “ScorePatients” but those peopleare not writing a patent specification at 2 AM.

ScoreVictims, in FIG. 10 self re-curses to run three sets ofcalculations.

The first time through ScoreVictims, it simply applies the POIlocations, captured earlier into an array, to those scoring anchors fromthe table ScoringChild to get the scores. For the four condition typesused in this example, each POI is only applied to the condition type ofthe person who's POI we are scoring the impact of.

In the second pass through ScoreVictims all calculations are based onthe POI's for subjects at one condition level being applied for scoringto the scoring anchors for the next lower condition group (a leftwardshift comparison). For example, a POI for condition number two isrunning against the scoring anchors that were developed using the POI'sfor condition number one. When we begin testing with subjects of unknowncondition, we will not want a person whose real condition is conditionnumber two getting scores from anchors associated with completelyhealthy users. This is a leftward shift calculation and will be used tosee how well the instant parameters being tested in the grand tourcalculate scoring anchors that prevent false positives by making theanchors for one condition level less sensitive to POI's from someonehaving a different condition level.

Similarly, in the third pass through, a rightward shift comparisonresults in, for example, a POI from condition number one being toosensitive to the scoring anchors developed for condition number two.

Upon completion of the third pass, and in the middle of FIG. 10B, asingle numerical score on 10 different measures of those comparisons ismade using implementer-adjustable levels of standard. A perfect score isa score of 10 which means that, in addition to the low-density anchorprevention of MakeAnchors, ScoreVictims verifies that each condition hassufficient hit magnitudes and that for each condition level there isseparation from the next lower and next higher level conditions toreduce bleed-over and false positives.

Thus, the optimization, in addition to verifying proper and substantialmagnitude of “hits” within a condition type, selects combinations ofparameters that best prevent viewers with one condition from scoringhighly on scoring anchors that are based on subjects with othercondition levels (only some of which are included in theOptimizationDataFBF table in FIG. 11).

Thus, FIGS. 11 through 13 illustrate subprograms for automatically andless subjectively creating potentially extensive scoring anchortemplates (covering any amount of the image) directly from evenextremely large POI datasets, automatically scoring POI's from bothsubjects with known conditions (for validation) and from subjects beingdiagnosed, and for optimizing the parameters that drive all of the abovein a process based on actual end results from iterative testing of wideranges of such parameters.

Finally, due to the capacity to handle very large numbers ofcombinations of even substantially disparate anchor areas that areostensibly unrelated to a single condition and the potential tonon-subjectively optimize these artifacts into groups of scoringtemplates, an enhanced ability to recognize and quantify less-obviousphenomena is possible.

More Discussion of Elements Described in the Above Code Snippets:

Yet another form of optimization, additive to and typically subsequentto the optimizations just discussed, for additional fine-tuning,involves the selective flagging for deletion of selected scoring anchorrecords that were acquired earlier in the programs of FIGS. 11-13. Inthe preferred embodiment of this extra fine tuning optimization, theoptimization values (obtained earlier by the GrandTour program and otherprograms shown in FIGS. 11-13 and stored in the tableOptimizationDataFBF) are later used to iteratively score POI's as isdone in the processes of FIGS. 11-13. However, in this last optionalstep, as the POI's of known subjects are iteratively scored, thatscoring is done with a different scoring anchor missing in eachiteration. It will be normative that this will tend to reduce ratherthan increase the score. However, applying the left shift right shiftprinciples for grading shown in the ScoreVictims program, it may befound that the remaining scoring anchors and result in better separationbetween the condition levels. That additional iterative process willenable the sequential removal of anchors detrimental to good separation.

If the removal raises the magnitude of the score or increases its scoredistinction from other condition scores this anchor (e.g., the databasetable ScoringChild) record is marked to be ignored and will not be usedfor scoring in the future. This iterative process continues until everyscoring template in every frame has been thus considered for separationfrom the scoring process while in the presence of all of the others.

Multiple Optional Scoring Strategies:

After the scoring templates have been developed from subjects with knownconditions, a new subject to be diagnosed can have his POI's capturedusing the exact same capture process. However, there are several optionsavailable to implementers in how to use this POI data. For example, thesingle set of POI data can be run against each condition's anchorscoring data individually to see which obtains the highest magnitudescore; the data group resulting in the largest score indicates the groupmost applicable to the instant subject.

Alternatively, since the condition one group has negative values forscoring anchors and the others have positive values, the subject beingdiagnosed may have his POI scored against all of the scoring anchors andsummed and the net magnitude of the positive and negative values fromapplying his POI's is his score or a product component of one. Thus, ifa person having positive characteristics does, in fact, look where ahealthy person would look, it will and should reduce his positive scorewith the responsive negative score component.

In this embodiment, it will not be unusual, when properly optimizedand/or optionally adjusted by implementers for better “channelseparation” (the data discretion between the condition levels) forcondition three scoring anchors to have higher values for target strikevalues than condition two scoring anchors, etc. Thus, in thisembodiment, a severely affected subject would likely, in addition toscoring high in his own condition category, traverse the scoring anchorlocations of lower condition scoring anchors as well. This will resultin an appropriately much higher score for a much higher subject tendencyin the direction of the condition being measured. Thus the magnitude ofthe score can be directly associated with the degree to which thesubject is affected by the condition being diagnosed.

There will also be many other applicable variations on these themesincluding hybrids such as a concatenation of the scores for the twoapproaches just discussed.

Strabismus:

Research unassociated with the study of Strabismus has also establishedthat there is a powerful cognitive and vision-system phenomenon known toaggressively facilitate conditions conducive to clear vision even whenthis requires brain-directed tasks that normally have nothing to do withvision perception. Thus, the brain appears to have a learning plasticitythat enables it to recognize indicators of and causative agents forvision clarity and to orchestrate immediate responses favorable to clearvision. These factors and the effectiveness of the comparativelyunpleasant current treatments suggest that applying deprived visionclarity as a response to instant failures of binocular synchronizationis an effective means for treating Strabismus. When that deprived visionis delivered by the current invention, it also has the advantages thatit can be applied early in the development of the vision system beforethe subject is verbal and can be tolerated for more extended periodswith less supervision.

Thus, strabismus is another example of a condition recognizable andtreatable by the current invention. As extensively described above andelsewhere herein, and following steps, some of which are outlined inFIG. 3 with programmatic support in FIGS. 11-13, subject POI's arecaptured in any of the viewing scenarios discussed herein as the viewerwatches.

The operations for dealing with strabismus can follow the same detailedsteps outlined in FIG. 3. However, for strabismus, step D(disorder/normative scoring) in FIG. 3 can be simpler than, for example,autism. Many eye-tracking systems, including ones like 108 in FIG. 1,provide, to a personal computer (like 109 which can run the controlsoftware) coordinate values relative to POI screen positions on thedisplay being viewed. Thus, the location on the screen being viewed iseasily known by simple scream position data. Some also calculate andprovide a representative vector of each eye's vision axis and even thedistance from a point between the viewer's eyes to the intersection ofthe viewer's vision axes and thus providing the distance to the viewer'spoint of focus.

Condition Recognition Factors: By Distance to Vision Axis Intersection

At this point, one of the most pleasingly easy to use embodiments of thecurrent invention is discussed. A subject, such as the young 101 in FIG.1, viewing a monitor 102 with POI at 103 being observed by theeye-tracking assembly 108 with results being captured by the controlsoftware running on computer 109. In FIG. 2 it can be seen that hisactual focus is behind the screen possibly because one eye has wanderedaway from a closer intersection with the visual axis of the other. Thesubject is seated at a known distance from the monitor. The controlsoftware either accepts the vergence-based distance from the viewer 101to the intersection of his vision axes that is provided by theeye-tracking assembly or calculates that same distance to the user'spoint of focus (POF) by very well understood trigonometry based on theangle of eye axes vectors provided by the eye-tracking assembly. Eitherway, that distance can simply be subtracted from the known distancebetween the viewer and monitor as a surprisingly useful indicator fordegree of strabismus.

Although this seems to be the simplest embodiment possible, it was foundto be unusually effective. As the lazy eye dysfunctionally moves evenslightly away from a normal vergence with the strong eye, a substantialdisparity rapidly emerges between the a priori distance between subjectand monitor and the distance to their current point of convergence. Thisis eminently recognizable by the difference between the a prioridistance and the distance provided by the eye-tracking assembly (orwhere unavailable the control software calculated distance to eye axesintersection). Thus it was found effective to simply set up a range ofcorrective stimulation responses based on distance.

Based on the now-known millimeters (mm) of “error” between the knowndistance to the monitor and the calculated range, a graduated level ofGaussian blur was applied responsive to the magnitude of that error.That graduated response was managed with five tiers based on the mm oferror. The first tier was 150 mm. if the error was less than 150 mm, nocorrective stimulation was applied.

If the difference between the a priori distance and that calculated wasless than 150 mm, no defocus was applied. If the difference was between150 mm and 175 mm

Similarly, for differences of 176-200 mm, 201-250 mm, and >250 mm,levels of defocus were increased until in the last group the image waseffectively imperceptible (nearly completely defocus).

Corrective Stimulation:

This embodiment is also exceptionally easy to provide at least somemodicum of corrective stimulation without writing a lot of software. Forexample, when the computer 109 is displaying the image with aQuickTime-based video player, the video may be blurred to a desireddegree by applying a Gaussian filter to every frame using, for example,Apple's Core Video technology for blurring me QuickTime-based videoimage.

Thus, by simply having the control software operating on 109 call acommercial program to blur the image being displayed relative to thenumerical magnitude of a past variable, the viewer's image wascontinuously responsive to the viewer's binocular fidelity.

As the lazy eye begins to move away, the distance calculation exposesboth the presence of binocular infidelity and a measure of itsmagnitude. A measure of this magnitude is used to determine the degreeof responsive blurring. Then, as the lazy eye returns to normal vergencewith the strong eye responsive to the inability to see the image, thedistance error decreases as does the degree of corrective stimulation.This provides an immediately apprehended perception by the viewer of acause and effect relationship between binocular infidelity (when the twoeyes just can't stay together at a mutual POI) and the inability to seethe video which is evocative of fusional vergence.

Other Recognition Factors of Binocular Fidelity and Other Indicationsfor a Positive Score:

There are a number of additive and/or alternative methods for rapidlyrecognizing the presence of binocular infidelity. Incorporating theseindications along with (in addition to) the above process can improveperformance and provide useful checks and balances. There is certainlyno advantage in a false positive and compliance is inverselyproportional to the number of frustrating false positives. Thus, in anadditional embodiment, multiple factors are used to recognize binocularinfidelity.

Eye elevation, observing one eye at a different relative cyclopicelevation than the other is an indicator of binocular infidelity. Thatis because, as is well known in Ophthalmology, the two eyes normallytrack together in some transverse plane. That is, the elevation of theleft eye is normally the same as that of the right eye with respect tothe cyclopic origin (which moves with the head). One benefit of thisoption (which is based on dysfunctional disparities in eye-elevation) isthat it requires no known distance to POI information. Another is thatit is additive to the other indicators for faster and potentiallybroader sensing of dysfunction. Also, is applicable to worn eye-trackingassemblies like FIG. 5.

Inactive/active partners: When one eye moves and the other does not,this is an indication of dysfunction.

Strong-eye leadership: It is typically already known which eye is thestrong eye and which is the weak. Based on this knowledge, it is anindicator of dysfunction when the strong eye begins or continues a paththat is or becomes incongruent with that of the weak eye. For example,if the strong eye follows a path divergent to that of the weak eye(increasing inter-pupilary distance or IPD) that may simply indicate amore distal instant POI. However, if the strong eye continues itsdirection and the weak eye's changes, this is an indication ofdysfunction.

Strong Eye Recognition:

It is normative for the strong eye to be an enduring condition. Thecurrent invention can recognize and indicate the strong eye.

The Eye that Best Follows Action:

As the current invention receives the eye tracking informationresponsive to images on the screen, for example in step C of FIG. 3, itcan be used to both identify the strong eye and approximate the instantdegrees of failure in mechanical fusional vergence (angle of squint). Byvery briefly placing an interruptive and discrete element on the screen(e.g., a brightly colored dot on a temporarily otherwise uninterestingbackground), the vision axis of the strong eye, as measured by the eyetracking equipment, will fixate upon and, when the locationally discreteelement moves, follow the discrete element better than the weak eye.Thus, in addition to other diagnostic functions of the control software,the option to identify the strong eye with the placement of a series ofinterruptive points or images on the screen followed by comparison ofwhich eye, as reported by the eye-tracking assembly, best fixates atthat point, is disclosed.

Games that Train and Measure:

Single Image; Requires No Eye-Separating Display

There are also other training and measurement games that do not requireeye-separating display.

One Ball Games:

Here, after the strong eye is identified as described elsewhere herein,a game background image is displayed on the screen and a single ball islocated at the intersection of the weak eye's vision axis and thescreen. (The location of the point of each eye's vision axisintersection with the screen is a common feature in eye-trackingcomponents and the overlay of an image at any given screen location overanother image is so widely understood that it is not recapitulatedhere.) The game (or in other applications the job such as moving acursor over an icon on the computer's main screen to execute a program)requires the viewer to move the potentially lazy eye so that its visionaxis intersects the screen where he wants the ball to go. Analogous toexercises where the strong eye is paralyzed or covered, this process, bydirecting the action on the screen only responsive to the lazy eye,forces it to perform in exercises that can be highly geometric and asprecise or flexible as desired by implementers.

Twin Balls:

The “twin balls” training and measurement game tracks each eye and,responsive to the intersection of each eye's vision axis with thescreen, locates a ball, icon, or other visual object at said points ofintersection on the screen. Thus, a healthy viewer with normalcorrespondence will see the two balls (or other visual object which arepresumed when “two balls” are mentioned) superimposed. One particularexample is illustrated in FIG. 4A. A game background for one such as amaze fills most of the screen. A goal of any kind is located at 402 anda single purple ball 403 is positioned to enter the maze and it staysthere for a few seconds. Then the ball slowly separates in FIG. 4B intoa blue ball 404 and a red ball 405 with the overlapping areas of the twostill remaining purple as the separation occurs to communicate therelationship between the balls. At this point, the blue ball is fixedbut the red ball now moves to and continues to track with the point ofthe intersection of the lazy eye with the screen. If the actualintersection of the lazy eye's vision axis with the plane of the screenis so far off that it is outside the display perimeter of the screen,and arrow, not shown in FIG. 4, is displayed whose tip points to thelocation and whose tail points to the blue ball waiting at the entrance.The tip is preferably near the point on the screen periphery that anextension of the arrow would cross and the length of the tail isproportionate to the distance between the blue ball and the point ofintersection of the vision axis of the lazy eye on the plane of thescreen (indicative of the magnitude of the desired correction stimulus).

As the viewer moves the lazy eye, the red ball 405 (in this example) thegame control software, responsive to the eye-tracking data for the eyepreviously identified as the lazy eye, moves the red ball on the screenin the direction of this change until it is close enough, within atolerance, to the position on the screen of the blue ball 404. If thetwo balls are within an implementer-chosen tolerance of spatialcoincidence, they become a single purple ball, if they merely overlap,the overlapping portion becomes purple. If the two balls are combined orare

balls are combined or are at least close enough together to fit throughthe entrance of the maze, the location of the blue ball on the screen isthen driven by the game-control software to now be located at theinstant intersection of the strong eye's vision access and the screen,thus allowing the viewer to direct with his eyes the progress in themaze. Of course, in this example, the balls are not allowed to cross aline (using well established software techniques for virtual-object pathcontrol through virtual boundary borders which are not recapitulatedhere).

If, during that progress, the two balls separate far enough apart thatthey are no longer able to fit together through a channel in the maze,the ball responsive to the position of the strong eye stays where it isand temporarily ceases to be responsive to positioning of the strong eyeuntil the lazy eye, still guiding its ball, returns that ball back towhere it is close enough to the now-fixed strong-eye's ball to fitthrough the maze. The game control software, when that condition isachieved, reactivates the relationship between the strong eye and themovement of its ball as it allows new progress for the two through thepaths, based on implementer-chosen parameters for adequacy of closenessof the two balls for passage between the channels.

The maximum and average disparities between the positions of the twoballs as well as the time required to progress the maze are parametersapplicable to diagnosis of the condition's magnitude as well as enablingthe benchmarking of progress over the period of rehabilitation. Also,unlike many of the quasi-static and slower response measures, approachessuch as these, particularly when timed, are of increased value both forthe development of higher-speed visual performance skills, and themeasurement for the capacity thereof.

Separated Images:

I Ball Through the Channel; Squint Angle Measurement:

An additional measure of vergence error in degrees can be used inembodiments where individual images are provided for each eye. Theseembodiments include any of the many approaches for providing separateimages to separate eyes. These include but not are limited to colorseparated glasses (e.g., the old red and green 3-D glasses),polarization separated glasses, shutter glasses, eye-individualizedheads-up displays and direct projection to individual eye display. Inembodiments where eye-image separation is accomplished through the useof worn optics that block the view of, for example, monitor-locatedcameras, the eye-tracking cameras are ideally located in the glassesthemselves. For example, in polarization or shutter glasses, e.g., 501in FIG. 5, a tiny camera 502 for each eye 503 in the frame of theglasses is preferred. Thus, at once, the wearer sees selective imagesfor each eye and the control software, which typically controls thedisplay, also simultaneously tracks where each eye is looking responsiveto said individual images.

Based on research of the past, it might seem logical to just project acentrally located cross on the video image (analogous to a Cartesianorigin in the center of the screen) visible to one eye whilesimultaneously projecting a dot visible to the other eye and prompt theuser to tell us the apparent location on the Cartesian axis through thekeyboard or voice recognition. While this, of course, is applicable toboth the assembly and the objectives of the current invention, there areother options provided by the current invention more applicable to lesscooperative (particularly pre-verbal viewers) and less patient (e.g.,teenage) viewers. As is explained elsewhere herein, the numbers andmagnitude of strabismic dysfunction can be identified and approximatelymeasured by the current invention by other methods discussed herein(including the measurement of eye-tracking vergence errors indicative ofa distance other than the known distance between the viewer and thescreen as well as non-orthophoric degrees and even directions ofadjustment, etc.)

However, an additional measurement strategy provides options applicableto corrective training, more measurement accuracy, less subjectivity,and no requirement for verbal or keyboard skills. One such embodiment ofthe current invention provides a brightly colored “ball” (602 in FIG. 6)on the screen 601 visible to only one eye by any stereoscopic displayapproach preferred by implementers). In one game environment, thebrightly colored ball 602 is at the bottom of the screen 601 and movesonly left and right at the bottom of the screen as the strong eye (whichis the eye that sees the ball) moves left and right. (The game softwaresimply places the ball towards the bottom of the screen and moves itleft or right so that its position on the screen laterally approximateswhere the strong eye's vision axis intersects with the screen.)Somewhere above the lateral track of the ball is the video image of achannel e.g., 603, target or other objective with an incentivized reward(which may simply be a musical tone and/or reward graphic indicatingthat access to the next video has been accomplished and will immediatelyfollow). For first-time users and those requiring more direction, ademonstration video first illustrates the ball 602 moving left and rightuntil it is perfectly aligned with the narrow opening to the channel (orother reward entry location) and then, after a brief delay, being movedinto the channel followed by the reward, e.g., instant music, image, orother announcement and a very brief but entertaining video, This isrepeated several times from several different starting points for theball with the same result until the viewer sees the relationship betweenthe ball position and a the reward. Then, the screen with the ballreturns but this time the lateral position of the ball on the screen,visible only to (due to stereoscopic separation) one eye (in thisexample will use the strong eye), is determined by the strong eye'svision axis. The relationship between the motion of the ball and theazimuth of the eye is quickly apprehended by the viewer. The rest of thescreen, i.e. typically everything but the ball (here only the boundarylines 604), is visible only to the other eye (here the weak or lazyeye).

Healthy viewers can quickly “get the ball rolling” to the desiredlateral point easily by looking from the ball to where they want it tobe. Thankfully, it is human nature, as we seek to “will” something tomove, that we seek to move it with our eyes. However, strabismic viewerswill have a problem. Even when the ball is precisely at the channel, itappears to the uncorrelated viewer to be off to one side relative, ofcourse, to the degree of strabismic error (squint angle). It would benice to know that angle and without requiring any viewer input.

Because the eye-training software that guides this process continuallyadjusts in real-time the lateral position of the ball on the screenresponsive to the calculated lateral point where the strong eye's visionaccess intersects the screen, as the viewer looks in the direction hewants the ball to go, the ball actually does move in that direction and,in doing so, also moves with respect to the actual screen location ofthe channel (using channel herein to represent any game target, etc.).When the ball stops and hesitates awaiting the reward, because in theadjusted binocular perception of the viewer the ball is at the targetlocation, the difference between the location of the strong eye'scurrent vision access intersection with the screen and the actualchannel location on the screen is one measure of strabismus angle duringan action sequence. (The mathematics of calculating this angle subtendedbetween the two points based on the distance from the user's eyes topoints on the screen, being well-known to all of those skilled in theart, is not detailed here.) Because the viewer is trained to anticipatea delay between the proper centering on the ball 602 (below the channel)and the incentive reward, the lateral distance on the screen between thepaused ball and the actual lateral center of the channel (not as seen bythe lazy eye but as actually presented on the screen) can provide avaluable measure of strabismus in a less quasi-static environment andwithout depending on data from verbal and/or subjective viewerimpressions and responses.

Initially, the game-playing viewer is given substantial tolerance sothat the reward can be obtained for positive reinforcement. However,using the game as a training mechanism for learned remediation, thattolerance can be gradually reduced over time proportional to anyimprovements over time to strabismus angle.

One example of an alternative vertical vergence measurement would be thehorizontal training game just described rotated 90° as shown in FIG. 6B(where, for example, the ball stays at the left of the screen 601 andinitially moves only up and down, and only enters the incentive channel603 to the right, leading to the next video or other reward, when theball 602 is at the level of the opening to the channel).

In both of the previous examples the views to each eye can be reversedin separate tests so that the weak eye determines the location of theballs 602 as they are displayed on the screen and the strong eye seesthe channel. By doing this both ways we both provide data on each eye'sperformance as well as capture data for both angle and direction ofstrabismus.

Strabismus and Autism Simultaneous Application:

Because the elements of the current invention are applicable even to thevery young and because both strabismus and autism emerge so early andboth require early response, it is advantageous that the currentinvention can be used to both diagnose and treat both at the same time.

In a preferred embodiment for one such application, the viewer watcheseither a prescreened and coded or real-time coded (e.g., any movie ortelevision program) video. For each frame, the viewer's eye-tracked eyepositions are considered both for their motor correspondence and forPOI's having autistic values.

When the viewer's POI strikes a positively-coded screen area (forconvenience, herein “positively coded” means a score indicative of thedysfunction being tested for), diagnostic autistic scores are capturedand tabulated as described elsewhere herein for later or real-timereporting on the presence, types, and magnitudes of autistic signatures.As in autism-only embodiments, the controlling software, by implementercontrol and/or user selection, can additionally apply remediativeaction. These actions may be chosen to be a blurring of the imagerelative to the magnitude of the positive score, vignetted highlight ofpreferred-focus areas, software-emulated diplopia, as well as any othersound or visible cues including the few that are discussed herein.However, in one preferred embodiment for combined autism and strabismusdiagnosis and remediation, software-emulated diplopia is not used as aremediation stimulation for autism so that it can be used forremediation of strabismus. This allows the potential plurality ofconditions being remediated simultaneously to have independentlyrecognizable stimulation cues. As in isolated autism remediation (whenstrabismus is not being considered), magnitude of the autism remediationstimulation cues are reduced proportionately as the viewer's scorebecomes less positive (for example, when his eyes come closer toengaging a negative target such as adult human eyes or other locationsas determined by implementers).

When the viewer's eyes, based on the numerous indicators for strabismusdescribed herein and others that logically follow, indicate strabismicbehavior, any of the stimulation cues may be implementer-chosen toprompt for remediation. While any of the cues may be used in anycombinations, they should be selected by implementers to beindependently recognizable.

For example, in one embodiment, the stimulation cues for remediation ofstrabismus is diplopia (that is, causing the presented image to beoverlapping images spatially separated from each other to look likediplopia where the degree of that apparent binocular mis-registration isresponsive to the degree of the strabismic behavior). The simultaneousprompt for correction of for autistic behavior can be vignetting. Here,the center of the vignetting effect can be the screen location of whereimplementers want the subject to be looking.

In programmatic terms, the display location where implementers want thesubject to be looking may be as simply described as theTargetCoordinates value (a data field described herein) of the negativescoring anchor with the highest magnitude for the instant frame. Thus,when the instant POI's are being scored against scoring anchors (e.g.,in step D of FIG. 3) and a highly positive score for autism occurs, thesoftware selects from the sorted scoring anchor table (e.g. theScoringChild example in FIG. 11) the record with the highest negativevalue for TargetStrikeValue and places the center of the vignetting atthe screen location indicated by the value of TargetCoordinates in thatsame record.

Once the general programmatic basis for a single-malady diagnosis andremediation, discussed at length herein, is understood, programming forcalculating both during the execution of a frame will be understood bythose experienced in the field of programming. This is useful wheremulti-malady diagnosis is desired such as with small children watchingtelevision. Those familiar with computer programming understand how tomake a sequential process look simultaneous by sequential processing andwill also understand how this can be accomplished using the anchorscoring table (e.g., ScoringChild) to score POI's against scoringanchors for recognizing autistic traits while, sequentially but duringthe same frame, considering and quantifying strabismic cues (asdescribed above) from instant eye-tracking data. Then, in step E of FIG.3, the image can, for example, both be vignetted (responses to anyautistic traits recognized) and made to appear more diplopic (responsiveto any recognized binocular infidelity).

Having said all that and having illustrated what is the inventor'spreferred embodiment for multiple malady diagnosis and remediation, itshould be noted that there are many applicable alternative programmingmethods and remediation cues that can be equally effective.

When vignetting is thus used, the periphery of the image around thepoint the subject should be looking typically gets darker and thevisible image in middle becomes smaller with both relative to themagnitude of the autistic behavior. When used as a cue for redirectionin autism, this is to direct the autistic subject's eyes to thepreferred location intuitively while preventing this direction frombeing mistaken as a correction cue for strabismus. Stimulation cuesshould be chosen to be intuitive.

Thus, it is actually possible to diagnose autism, strabismus, and otherconditions simultaneously. It is also possible, where desirable, totreat them simultaneously. The viewer's eyes are drawn to the preferredattention area respective to autism while simultaneously requiringproper motor correspondence to see a sharp image.

Head-Worn Eyetracking

For head-worn eye-tracking devices, after an initial calibration so thecameras know where the eye and/or eye-reflection landmarks are in thecamera view,

3.4. Strabismic Display Imaging (SDI):

Emmetropic glasses, as detailed below in 3.5.1, have the potential forcreating a defocus condition in the presence of binocular infidelity.

For non-presbyopic strabismics with otherwise healthy eyesight, however,the automatic defocus of emmetropic glasses as applied to strabismusrequires further design constraints. The accommodative capacity of theotherwise healthy strabismic requires no correction. Thus, there is nofocus correction applied responsive to a change in calculated distance.In other words, in the presence of substantive natural accommodationthere is no need for the distance-driven external accommodation (thatconveniently provides defocus responsive to binocular infidelity).

Since this does not affect the SDI approach used in this research (SDIis not affected by this since its defocus is screen-driven) and thus hasno evaluation in the current research, it is covered here only verybriefly. Future research will determine which methods are the mosteffective for the emmetropic glasses embodiment.

A variety of processes may be used to narrow the DOF of the subject andmove the center of that band with respect to the calculated POI. E.g.,for a subject requiring no focal correction, the emmetropic opticsthemselves may be directed to offset sensed and/or calculated/predictedeye lens diopters to artificially offset accommodation. Alternatively,the accommodative capacity may be limited in a distance driven manner byeffecting, with the optics of the emmetropic glasses (which are passivein the absence of binocular infidelity) a degree of myopia or hyperopiathat places the calculated POI (based on actual eye-vector intersection)in focus. If, however, the true POI is not at the point of intersectionof two vision axes (binocular infidelity), a defocused conditionresults.

Additively, the distance-independent vector indicators in 3.5 can beused independently and may also be used in coordination with the aboveprocesses to ensure a richer, more sensitive system.

3.5.2. SDI Management of Binocular Fidelity:

SDI's indication of dysfunction is, perhaps, more easily understood thanthe emmetropic model. Also, the list of distance-independent indicatorsin 3.5.1 is also applicable to SDI. SDI determines Strabismicdysfunction from a priori monitor distance related to calculated POIrange. Thus, binocular fidelity is currently scaled by the differencebetween the actual distance to the monitor and the distance indicated bythe intersection of the two vision axes, which is itself driven by thetwo eyes' azimuths.

Also, in future research the distance to the monitor itself will becorrected by the ranging system's ranging data averaged over time withanomalies above a threshold removed. It is believed that most subjects'eyes will behave enough to estimate the true current distance to themonitor over time. However, in all of our very preliminary testing todate, we have simply compared calculated POI range with an approximateknown distance to the monitor.

This desirable state (appropriate focus support) is effected by anelectro-optically varied focus in the lenses of the emmetropic glassesto be added after the initial ranging portion of the research iscomplete. (Other dynamic correction mechanisms are, of course,applicable but electro-optics are used here as an example embodiment.)However, in the presence of Strabismic dysfunction, one eye's departurefrom the other eye's POI will result in a different (normally verydifferent as one eye wanders off) intersection of the two vision axes ofthe two eyes. If, for example, the subject is talking with someonenearby (fixating on their eyes) and the right eye then detaches from thePOI (e.g., pans to the right in a normatively significant degree), theintersection of those two vision axes will move from a couple of feet towell behind the true POI. As a result, the natural emmetropic processwill demand a lens diopter setting for a distance distal to the POIresulting in defocus. Only by returning the lazy eye to binocularsynchronization can the brain regain clear vision through anow-corrected emmetropic optic.

What is claimed is:
 1. A device for at least identifying the state of auser with respect to a condition comprising: an eye-tracking assembly tolocate where the user is looking; a camera configured to capture imagesof the view before the user; an image element recognition componentconfigured to locate in said images recognizable elements known to beassociated with, if a user looks at them, a state with respect to thecondition; and a processing assembly configured to determine, from saideye-tracking assembly and said image element recognition component, atleast if the user looks towards a said recognizable element in said viewto produce a score at least indicative of the user's state with respectto the condition.
 2. The device of claim 1, wherein said camera is avideo camera; wherein said device is enabled to function effectively inreal time.
 3. The device of claim 2, wherein a said score is calculatedfor a frame from said camera at least when a said user looks towards asaid recognizable element.
 4. The device of claim 1, wherein themagnitude of said score is responsive to the distance between thelocation of a said recognizable element in an image and the location inthat image that correlates with where said user looks, at least within adefined range of distances; wherein a said score responsive to wheresaid user looks is effected, at least when said distance is within adefined range of distances, even when said user didn't look precisely ata said recognizable element.
 5. The device of claim 1, furthercomprising: data, for at least one said recognizable element, availablein the configuration of said processing assembly, at least indicative ofthe effect someone looking at that recognizable element should have onsaid score; wherein the magnitude of said score is responsive to whichsaid recognizable elements said user looks at.
 6. The device of claim 1,wherein said processing assembly considers a plurality of saidrecognizable elements that said user looked towards to produce a resultthat is at least indicative of said user's state with respect to saidcondition.
 7. The device of claim 1, wherein said images are videoimages and can include images, captured by said camera, of a video beingdisplayed; whereby the device can function while said user is viewingany displayed media.
 8. The device of claim 1, further comprising asound component producing, responsive to said score, at least one of 1)a sound indicating at least that said condition is presenting, 2) asound indicating at least that said condition is not presenting, 3) asound indicative of the degree of the recognized presentation of saidcondition, 4) a sound indicative of the degree of the recognized absenceof presentation of said condition or 5) a verbal message.
 9. The deviceof claim 1, further comprising: a display, located and aligned to beperceived as at least partially superimposed with at least part of saidview, to be directed by said processing assembly to provide at least oneof 1) at least one displayed indicator indicating where said user shouldlook based at least on the locations of said recognizable elements knownto be associated with a healthy state, that is, free of said condition,2) at least one displayed indicator indicating where said user shouldnot look based at least on the locations of said recognizable elementsknown to be associated with an unhealthy state, that is, a presence ofsaid condition or 3) a temporarily modified image to encourage a changein viewer behavior responsive to a said score indicative of a presenceof said condition.
 10. The device of claim 1, further comprising: a viewmodifying component between the eyes of said user and said view causingat least one of 1) a degraded view of said view responsive to a saidscore indicative of a presence of said condition or 2) making areas ofsaid images, where there are said recognizable elements known to beassociated with the presenting of symptoms of said condition, lesseasily visible than other areas.
 11. A device for at least thepreliminary diagnosis of a condition of a user comprising: a display; atleast one image to be displayed on said display to the user; aneye-tracking assembly to locate where the user is looking; an elementrecognition component configured to locate, in said at least one image,recognizable elements known to represent at least one of the groupcomprising: things that those with the condition tend to look at; thingsthose without the condition tend to look at; and a processing assemblyconfigured to determine at least if there is a presence of the conditionwhen the user, according to said eye-tracking assembly, looks towardssomething that said element recognition component identifies in said atleast one image as a said recognizable element.
 12. The device of claim11, wherein said processing assembly determines a degree of saidcondition indicated in said user based on criteria chosen from thegroup: if a said recognizable element that said user looked towards isone that those with the condition tend to look at; if a saidrecognizable element that said user looked towards is one that thosewithout the condition tend to look at; a known value for quantifying theeffect on said degree of a user looking at the said recognizable elementthat said user looked at; a known value for quantifying the effect onsaid degree of a user not looking at a said recognizable element thatsaid user did not look at; how close a said recognizable element is tothe location in said at least one image where said user looked;accumulated values from a plurality of said recognizable elements thatsaid user looked towards.
 13. The device of claim 12, furthercomprising: a sound component producing sounds chosen from the group: asound indicating at least the presentation of said condition; a soundindicating at least that said condition is not presenting; a soundindicative of said degree of the recognized presentation of saidcondition; a sound indicative of said degree of the recognized absenceof presentation of said condition; a verbal message.
 14. The device ofclaim 11, wherein when said user looks within a defined distance of asaid recognizable element, said processing assembly determines a scorethat is at least indicative of to what degree where said user is lookingindicates the presence of said condition based at least on criteriachosen from the group: how close where said user looked is to thatrecognizable element; if that recognizable element represents thingsthose without said condition look at; if that recognizable elementrepresents things those with said condition look at; a determined valueat least indicative of how much a look at said recognizable elementshould affect said score.
 15. The device of claim 14, furthercomprising: an image modifying component which is instructed by saidprocessing assembly to, responsive to a said score that is indicative ofa presence of said condition, cause said display to present images thatare less easily viewed, at least in their entirety, at least until ascore that is less indicative of a presence of said condition isdetermined.
 16. The device of claim 11, wherein said processing assemblycommunicates with said user with things chosen from the group: at leastone displayed indicator on said display indicating where said usershould look which includes at least the location of at least one saidrecognizable element known to represent things that those without saidcondition tend to look at; at least one displayed indicator on saiddisplay indicating where said user should not look which includes atleast the location of at least one said recognizable element known torepresent things that those with said condition tend to look at; areduction in the image visibility seen on at least parts of said displayat least when said user looks at a said recognizable element known torepresent something that those with said condition tend to look at. 17.The device of claim 11, further comprising: an image modifying componentwhich is instructed by said processing assembly to cause said display topresent images to be less easily viewed, at least in parts of itsdisplay area, at least in temporary response to said user looking at asaid recognizable element known to represent a thing that those withsaid condition tend to look at.
 18. The device of claim 11 wherein saidprocessing assembly calculates a result indicative of said user's statewith respect to said condition based at least on a plurality of saidrecognizable elements that were looked towards by said user.
 19. Thedevice of claim 11, further comprising: a video camera to provide saidat least one image; whereby said device is able to function effectivelyin real time with any image content that the camera photographs.
 20. Thedevice of claim 19, wherein at least said display portion of said deviceis worn by the user; and said display aligns said image with the normalview before said user; whereby said user perceives objects in said imageto be where they actually are.
 21. The device of claim 11, wherein saidat least one image is a video image; whereby the rapidly provided seriesof images enables the functionality of the device effectively in realtime.
 22. A method for at least identifying the presence of a conditionin a user comprising the steps of: acquiring at least one image to bedisplayed; displaying an image; identifying things in the image believedto suggest, if the user looks at it, at least if a presence of thecondition is indicated, identifying if the user looks towards one ofthose identified things, scoring at least if the user appears to havesaid condition based on if the user looked towards one of thoseidentified things, which results in a score; continuing for more images,when desirable, at the step of acquiring at least one image to bedisplayed.
 23. The method of claim 22 wherein the step of displaying animage is after rather than before the step of identifying things in theimage believed to suggest, if the user looks at it, at least if apresence of the condition is indicated.
 24. The method of claim 22wherein said score is cumulative in consideration of a plurality ofimages thus displayed and scored for the user.
 25. The method of claim22 further comprising the step of modifying at least the ease ofvisibility of said at least one image responsive to said score with thisnew step occurring before the step of continuing for more images, whendesirable, at the step of acquiring an image to be displayed.